Wintellect  

Roberto Farah and Brad Wilson figured out the undocumented, but extremely cool .cmdtree WinDBG command. It lets you create a custom WinDBG window that contains any commands you want displayed in a tree control just waiting for you to double click on them. It makes using WinDBG a lot less painful, especially for SOS debugging.

Since I do a lot of SOS debugging, I created the following .cmdtree file to make using SOS a little easier. I've also included Steve Johnson's great SOSEX extension as well.

windbg ANSI Command Tree 1.0
title {"SOS Commands"}
body
{"SOS Commands"}
 {"Load SOS and SOSEX"} {".loadby sos mscorwks;.load sosex"}
 {"Modules"}
 {"All Modules"} {"lm D sm"}
  {"Loaded Modules"} {"lmo D sm"}
  {"Loaded Modules (verbose)"} {"lmvo D sm"}
  {"Modules w/o symbols"} {"lme D sm"}
 {"Threads"}
  {"Managed Threads"} {"!threads"}
  {"Native Threads"} {"~"}
  {"Thread Pool Threads"} {"!threadpool"}
  {"Sync Block Deadlocks"} {"!dlk"}
 {"Stacks"}
  {"Current Managed Stack"} {"!clrstack"}
  {"Current Managed Stack Parameters and Locals"} {"!vars -w"}
  {"Current Native Stack"} {"kP"}
  {"Current Dump Stack"} {"!dumpstack"}
  {"All Managed Stacks"} {"~*e!clrstack"}
  {"All Managed Stack Parameters and Locals"} {"~*e!vars -w"}
  {"All Native Stacks"} {"~*kP"}
  {"All Dump Stacks"} {"!eestack"}
 {"Memory"}
  {"Stack Objects"} {"!dso"}
  {"GC Heap"} {"!eeheap -gc"}
  {"Heap Stats"} {"!dumpheap -stat"}
  {"Objects > 500 bytes"} {"!dumpheap -min 500"}
  {"Objects > 1000 bytes"} {"!dumpheap -min 1000"}
  {"Gen 2 Objects"} {"!dumpgen -stat 2"}

Because the file format the .cmdtree command is fairly fragile, you can download the above file here. When it comes to creating your own .cmdtree files, always makes sure to leave a blank line at the end of the file or the .cmdtree command can't load the file. Also, only use one space for indenting and separation.

After you fire up WinDBG, to show the Window, execute the following command:

.cmdtree c:\util\sos-cmdtree.txt

In playing around with the .cmdtree command, I noticed that WinDBG happily lets you execute the command multiple times so you can open up different sets of command files. While SOS will always be a little painful, with the .cmdtree command you don't have to type as much anymore.

Now that the .NET Reference Source Code for .NET 3.5 SP1 has made it up to Microsoft's servers, Kerem Kusmezer and I updated NetMassDownloader to support populating your symbol server cache directory with all the source code and symbols. Microsoft did some tweaks to the debugger with SP1 and the symbols are expected to be in a MicrosoftPublicSymbols directory under your symbol cache in order to download the reference source files in the debugger. I also cleaned up the program output, fixed a problem where NetMassDownloader wasn't delete temporary files correctly, and handled the case where the download PDB files have source server information but is missing the HTTP download info.

If you've used a prior version of NetMassDownloader, it's probably best if you delete your cache directory and start fresh. Also note that NetMassDownloader 1.5 assumes that if you are using VS 2008 you have applied SP1. VS 2005 and CodeGear are still supported as they were before.

If you're like me and want to use both the reference source code server (http://referencesource.microsoft.com/symbols) as well as Microsoft's normal public PDB server (http://msdl.microsoft.com/download/symbols) so you can get the public symbols for the operating system and the other things you debug, you'll find that the debugger downloads the symbols to both your symbol cache directory and the MicrosoftPublicSymbols directory underneath it. Since not everyone uses both Microsoft symbol servers, I didn't add the support to NetMassDownloader to copy the symbol files from the MicrosoftPublicSymbols to the main symbol server cache. I manually copied the folders after running NetMassDownloader and the debugger is happy.

With the recursive directory support in NetMassDownloader, it's trivial to grab all the .NET 3.5 SP1 reference source. Here's the command line I used on Vista x64 to grab both the 32-bit and 64-bit .NET binaries:

Netmassdownloader –d c:\windows\Microsoft.NET –d "c:\Program Files\Reference Assemblies"

With an empty symbol cache, it'll take a while so you might want to let the command run over lunch. If it's still running when you get back, there's always Solitaire.

As always, please file bug reports and features request as the NetMassDownloader page on CodePlex.

Eons ago in Internet years, I wrote what proved to be a very popular tool, Tester, the latest edition appeared in my second book. With it you could record and playback UI automation scripts. While Tester wasn't perfect, the number of companies using it as their automation tool of choice was very gratifying. In fact, there were so many, I probably should have made a business out of updating and supporting the tool. At least it was very gratifying to know I'd produced something useful!

Since I rarely do any UI development, I don't need Tester that much myself. Today, I had a need so grabbed the code with the thought of giving it a quick dusting off so I could use it on Vista x64. After 20 minutes of some tweaks and adjustments to move the application to 64 bits, the real work started. My debug builds were always asserting because SetWindowsHookEx was returning false when my recorder application tried to set the WH_JOURNALRECORD journal recorder to record all the mouse and keystrokes. The last error value was access denied so I simply thought that I needed to run the recorder process with elevated rights. If only it were that easy, because I still got the access denied when running with high integrity as well.

It turns out lots of people were reporting the same issue, but all the reports were from way back in the Vista beta days. After several hours of searching I'd run across several articles where there were discussion about how the journal recording hook was used by key loggers and other nefarious programs so Microsoft was making changes in Vista to close that hole. While I was all in favor of closing the hole, how in the heck was I going to get my simple recorder application to work?

According to some of the forum reading it looked like people got journal recording working by turning off UAC. That was totally unacceptable to me as I have written before I like UAC. Anyway, turning off UAC just for journal recording was the easy way out and made me feel dirty. I dug in for battle and fired up the search engines hard. As I never found this information in one spot, I thought I'd put it here in case someone else wants to use a journal recording hook.

  1. Your program's manifest has to be set to requireAdministrator for the execution level. Because a journal recording hook falls into the accessibility portion of windows, the uiAccess must be set to true. Here's how you'll set those up in the Linker, Manifest File section of the property page:
  2. The output binary must be digitally signed with a valid certificate. Read about how and where you can get reasonably priced certificates here.
  3. You won't be able to run your application where you built it. This was the hardest item for me to figure out and it wasn't until I ran across a random forum entry that provided the missing clue. You must copy the binary that sets the recording hook into a directory under the C:\Program Files directory. That's the only acceptable place for accessibility applications to run. By copying the binary, you can run the application and get your hook set. Of course, if you are going to distribute your application, you'll need to have your installation put it in the C:\Program Files directory and not allow the user to change it.
  4. You won't be able to start your binary under the debugger even if it's properly located in the C:\Program Files and you've started the environment with elevated rights. Visual Studio will give you the following error:

    Unfortunately, clicking on the Help button brings you to a "Information Not Found" page so it's a mystery. For you WinDBG users out there, the uiAccess set to true bothers WinDBG as well if you try to start the application from the debugger:
  5. To debug your journal recording application, you'll have to start it and attach the debugger. You'll want to make sure your application gives you a chance to attach before you start doing a lot of work.

Since this took three hours of my life to figure out I thought I'd share the solution in case someone else needed the information. While it would have been nice to have all this information in the journal recorder documentation, I'm really glad Microsoft shut down the security hole!

As the world knows, Visual Studio 2008 SP1 is out so start your download engines. I've installed it on one machine so far and I thought I'd mention a few things I found interesting.

  • The install took exactly an hour for me and was far better than the Visual Studio 2005 SP1 torture. After the first Next click I just let it run and didn't have to baby sit it. I installed off the ISO version as I didn't want to take any chances. I wanted all the bits with me as I was more afraid of a partial or corrupt install with the "piece at a time" download. The machine I installed on had the Team Suite edition so I knew I'd need everything anyway.
  • While many Microsoft folks have blogged about their team's particular adds to SP1, you can find the complete list of added features and bug fixes in Knowledge Base article 945140. Based on the number of features added, the version number really needs to change. However, I'm not complaining!
  • All my .NET applications certainly feel much faster because of the .NET Framework 3.5 SP1 speedups. I wish I would have taken some startup timing statistics because even WPF applications start reasonably quickly now.
  • Visual Studio has a new icon! I guess that will make it easier for us to tell the difference between Dev 9 and the rapidly approaching Dev 10.
  • Is there a story behind the option dialog UI change? Notice that the checkboxes are now flat instead of 3-d.
  • I love the new debugger option to cancel Symbol Server PDB file downloads. This is a great option on slower connections:
  • Another favorite debugger feature is that on managed code the debugger looks when you are stepping into a method and if it's a small property or operator, it turns the step into a step over. Now you don't have to apply the DebuggerStepThroughAttribute to all your one line methods.
  • Now the debugger knows about the .NET Reference Source Code out of the box. Unfortunately, as far as I can tell, Microsoft has not posted the correct PDB files for the .NET 3.5 SP1 release. There are PDB files up on the public symbol servers, but they do not contain any of the Source Server indexing. The previous version of System.PDB was 6.68 MB. Now it's 595 KB and definitely contains none of the appropriate reference strings. Also, my NetMassDownloader just returns Bad Request when trying to access the code. I'll let you know what I find out from Microsoft.
  • My personal favorite new feature is that Step Into Specific has finally made it to managed code. Say you have a line of code like the following:

    OuterMethod ( InnerMethod ( ) );
    Prior to SP1, if you did a Step Into, you always stepped into InnerMethod whether you liked it or not. Now, if you want to step into OuterMethod, right click on the line and behold the magic of Step Into Specific:

  • The TFS updates are totally dreamy. Brian Harry has the complete list here.

On a particular Vista machine, I was setting up a PerfMon view to monitor the .NET memory performance of an application and ran into a problem. The.NET Memory # Bytes in All Heaps Performance Counter wasn't in the list of performance counters. That was quite odd so I checked if that Performance Counter was there in Process Explorer by adding it to the main window display. Since Process Explorer has all the Performance Counter names hard coded, that would tell me if the problem was with the Performance Counter Library itself as PerfMon builds the list dynamically.

After pressing OK, Process Explorer showed no values at all in the columns. That pretty much proved that there was something seriously wrong with the Performance Counters on that particular machine. Without those performance counters it's pretty hard to do monitoring and performance tuning! Comparing two machines side by side, I noticed that the problem machine was missing a metric ton of other Performance Counters.

A few quick internet searches got me to the Microsoft KB article 300956, which discusses how to manually rebuild Performance Counter Libraries. If you read over that article, you start to get very scared very quickly because it tells you to manually replace files on the machine with the files from the installation CD, a bunch of manual registry editing, and so on. Fortunately, at the end of the article, they mention that running LODCTR /R from an elevated prompt (note the /R uppercase is required) will rebuild the Performance Counter Library including third party and extension performance counters.

Figuring I had nothing left to lose, I gave it a run. After shutting down PerfMon and Process Explorer, I restarted both and was very happy to see all the Performance Counters reappeared. After a big sigh of relief, I thought I'd write up what worked in case you are missing some Performance Counters. Of course, this advice comes with the usual Internet warning: Worked on My Machine ™.

So there I was knee deep in a great debugging challenge. On one machine, I had set up somewhere around 15 advanced breakpoints that had all sorts of properties set such as conditionals, hit counts, and so on. As I was feeling like I was getting close to the bug, I wanted to set up an identical debugging session with those breakpoints on another machine. While I could have manually recreated all those advanced breakpoints, I tried to cheat. Your breakpoints, along with Watch window variables, window layout, and other session state items are stored in a hidden file, <Solution Name>.SUO. I copied that file over to the other machine and opened the solution file. Visual Studio promptly crashed.

The .SUO file is the bane of your existence. Nearly all the problems you encounter with Visual Studio are the result of a corrupt .SUO file. Sadly, it seems all it takes to corrupt the .SUO file is your heart beating. In other words, whenever you have Visual Studio crash, refuse to debug, or behave strangely it's the .SUO file's fault. Whenever anyone asks me about strange Visual Studio behavior, my instantaneous response is "Delete the .SUO!" When it comes to Visual Studio hygiene, deleting the .SUO file is right up there with cancer screenings.

As I am tired of manually setting my breakpoints again on other machines and sick of the .SUO corruption bug I decided to solve it once and for all. I wrote a Visual Studio 2008 add-in, Debugger Settings, which makes it easy to save and restore your solution's breakpoints. It also makes it easy to create and apply different breakpoint sets on a solution, which is something I've wanted in Visual Studio for a long time. You can download the code for Debugger Settings here.

Debugger Settings is at version 1.0 and I would love to hear your feedback. Please note that I've only done my testing on Visual Studio Team Suite, but Professional Edition and above should have no trouble. Debugger Settings will not work with the Expression versions as they do not support extensibility.

Debugger Settings currently only saves and restores breakpoints but does work with both .NET and native C++ projects. I would love to be able to get and set the Watch window variables, but it does not look like that can be done without extensive reverse engineering. However, if there's enough demand, I might be persuaded to give it a go. Another feature I'd like to add is the ability to save and restore custom Exceptions added to the debugger's Exception dialog. Please let me know if you have other features or find problems.

Installing Debugger Settings

  1. After downloading the zipped code, unzip it keeping the directory structure intact.
  2. You may not need to do this step, but I've always had problems with Visual Studio not loading my add-ins correctly if I put them in the defalt Documents\Visual Studio 2008\AddIns directory. That may be because I've got my Documents directory in a domain redirected folder. If you get a FileNotFoundException with a code of 0x80131047 attempting to load Debugger Settings, you'll want to add the <unzip dir>\DebuggerSettings\Debug directory to the Add-in file paths by going to the Options dialog, Environment, Add-in/Macros Security dialog:
  3. Because Visual Studio has a problem finding assemblies in the same directory as the .AddIn file, you'll need to edit <unzip dir>\DebuggerSettings\Debug\DebuggerSettings.AddIn and in the Assembly elements, enter the complete path to DebuggerSettings.DLL.
  4. Restart Visual Studio.

Using Debugger Settings

Debugger Settings works mainly in the background, but you can set its options in the Options dialog.

By default, Debugger Settings will save any breakpoints you have set when you close a solution to a .DebuggerSettings file in the same directory as the solution. When you open a solution, Debugger Settings will look for the .DebuggerSettings file and reset any breakpoints that were not set from an existing .SUO file.

Debugger Settings adds four commands to the IDE, which should be self explanatory:

  • DebuggerSettings.RestoreFromFile
  • DebuggerSettings.RestoreSolution
  • DebuggerSettings.SaveToFile
  • DebuggerSettings.SaveSolution

As an example, to save a set of breakpoints to a named file, you'll go to the Command window and execute the DebuggerSettings.SaveToFile command. If you don't specify a filename as the parameter to the command, you'll be prompted for the filename with the File Save dialog.

There are four types of breakpoints that Visual Studio supports: file, function, address, and data (for native C++ development). My original intent was to support all the breakpoint types, but when you attempt to set an address or data breakpoint through the Visual Studio automation model, the breakpoint is immediately set to disabled. Hopefully a future version of Visual Studio will let us automate setting those two breakpoints. The good news is that file and function breakpoints are the huge majority of breakpoints. I implemented the work to set address and data breakpoints in case this gets fixed in the upcoming Visual Studio 2008 Service Pack 1.

One other limitation of the Visual Studio automation model is that there's no way to know if a Tracepoint has the message disabled. The Breakpoint2.Message property is always set to the default string value. Thus, if you have a Tracepoint set to execute a macro, but not output the trace string, as shown below, Debugger Settings will enable the Print a message checkbox simply by setting the Message property.

Wrap up

The implementation of Debugger Settings is very straight forward. The only moderately interesting technique I used was to work around the fact that the Breakpoints.Add method does not support setting Tracepoints directly. Visual Studio's code metrics reports only 467 lines of code so far in Debugger Settings. If I can figure out how to get the Watch window variables, I'm sure the code size will grow exponentially.

Let me know either in comments to this entry or through email (john at this company's domain) if you find Debugger Settings helpful or find problems. Also, any feature requests strongly considered.

If you're using WiX 2.0, my Paraffin tool can help you automate some of your setup chores. To read more about Paraffin, see the three part article, Part 1, Part 2, and Part 3.

Thanks to Matthew Goos who sent me the patch, there's a new –dirref when creating the initial .WXS file. That allows you specify the DirectoryRef element Id value. The default is INSTALLDIR.

When updating an existing file, I now allow you to specify both –ext and –direXclude switches so you can add additional extensions and directories to ignore. Any new ignored extensions or directories are added into the output file comment section so they are there the next time you update.

Thanks to all of you who have downloaded Paraffin. If you're using Paraffin, please let me know as I'm curious if others find it useful. Numerous people have asked if there's a WiX 3.0 compatible version on Paraffin. Right now there isn't but I'll update Paraffin to support WiX 3.0 as soon as the WiX team declares a stable 3.0.

Grab the latest Paraffin here. As always, let me know if there are any features you'd like to see or bugs that need squashing.

Perception is everything. That applies to everything in life and certainly applies to Vista. It doesn't take much web searching to find plenty of commentary indicating that Vista has not been well received and is often downright hated. In a lot of ways, I think Vista's been given a bad rap, and my opinion is that it is quite a bit better than it gets credit for. In this article I want to talk about what I think Microsoft did right with Vista. Is it perfect?--of course not, but no software ever is. Additionally, I want to address where I think many of the Vista problems are coming from. Finally, I want to also discuss what I see are the challenges Microsoft faces with Win7 given the Vista perception and how Microsoft can negate them.

Before I start, I need to say that I'm definitely not either a marketing person or much of a business person. I'm software developer who's bet my whole career on developing for Microsoft operating systems. I started with DOS 5.0 and switched to Windows 3.0 as soon as it came out. While some of you will say that makes me old, I prefer to think that means I have a long term perspective on Microsoft and its position in the developer mindshare. As a reminder, these opinions are my personal opinions.

Vista, especially after SP1, is definitely better than Windows XP. Every time I have to use Windows XP or Server 2003, I find myself really missing some of the features and abilities I've grown to love in Vista. As I was reading yet another screed against Vista, I jotted down the six key things I find invaluable about Vista. There are many more, but these are the ones that are important to me as a user and developer.

  1. User Access Control (UAC)

    The most important point Vista has is that it is the most secure operating system Microsoft has ever released. The reason for that is UAC finally keeps people from running with administrator credentials by default. Having run as a non-admin on XP and Server 2003 for nearly nine years, I'm much happier than I was being forced to use the RUNAS tricks. I can honestly say I've never found the UAC pop ups annoying or too frequent.

    While many people have complained that Microsoft should have implemented something like UNIX security, I think UAC is the best compromise for the reality of the Windows world. Microsoft has to deal with a ton of poorly written software from third party developers who assumed everyone was an admin just like they were. While some of this fault lies with Microsoft, a majority of it belongs to developers who won't or can't follow the design guidance. Microsoft has been issuing recommendations and steps for years on how to correctly write software, but developers certainly haven't been listening.

    What always bothers me is when I see someone recommending that you turn off UAC. If you turn off UAC and run all processes with administrator rights, you deserve all the viruses and slowdowns you'll get. Microsoft does a good deed working to make the OS secure and these "experts" go off and completely ruin the security. I wish Microsoft had not allowed UAC to be turned off.

     

  2. Mobile Computing

    Another Vista feature that makes my life better is that mobile computing finally works consistently and reliably. Back when I had Windows XP and Windows 2000 on my laptops, it was always an adventure opening a sleeping laptop. You just never knew if it was going to wake up. With Vista on multiple laptops, it's just works.

     

  3. Search on the Start Menu

    The search in the Start menu is simply brilliant. I've become addicted to it and whenever I go back to Windows XP or Server 2003, I'm always stumbling over how to open files or start programs. The OS now gets out of my way and doesn't interrupt my thoughts nearly as much.

     

  4. Ink Input

    As a Tablet PC fanatic, I love how Vista has made ink and the pen first class input for the OS compared to how it seemed to always be a tacked on feature in previous releases. I work best by writing things down by hand and Vista's trainable handwriting recognition means I easily get 98% to 99% accuracy when searching for things in my beloved OneNote. It's very sad how few people have gotten to use a Tablet PC because Bill Gates is right: a pen is a wonderful way to interact with a computer.

     

  5. Better Scalability

    In my opinion, Vista scales better than previous Microsoft operating systems. In the past, I was always in the habit of shutting down applications unless I was actively using them. As a developer, I always gave most of the machine to development tools because the more programs you had running the slower everything got. With Vista, I'm rarely shutting anything down and running tons more applications. Right now on this Vista x64 machine there are five instances of Visual Studio 2008, five PowerShell windows, Winamp with G-Force, three Internet Explorer sessions, OneNote, Outlook, Word, Process Explorer, and those are the just the open windows. I do more on a machine than 95% of users do, and Vista handles it easily.

     

  6. Fewer Reboots

    Finally, I never reboot. What's the old developer answer to everything on Windows? Reboot. Back on Windows 2000, I rebooted three to five times a day. On Windows XP, I generally rebooted every day. With Vista, I only seem to reboot on patch Tuesday. Both my desktops and notebooks running Vista go weeks between reboots.

There are many other positives to Vista, such as Media Center, and so on. Having felt Vista was pretty good, I wanted to give it credit given all the negative reports floating around. I'm definitely not discounting the bad press, because a lot of it is true. What I want to turn to now is where I think the blame lies for the perceptions around Vista.

Many of the problems with Vista have been entirely self-inflicted by Microsoft. From the beginning, they "over promised and under delivered." What it looks like to me is that almost nothing we were promised at the 2003 Longhorn PDC was actually delivered. The stream of announcements saying this feature and that feature were being cut started the Vista perception long before it ever shipped. I know I'm stating the obvious here, but I think it's one of the major issues that affects perception with Vista.

The whole "Vista Capable" fiasco certainly didn't help. The story is that Microsoft labeled computers with Intel integrated graphic chips as able to run Vista, when they clearly could not. (Am I the only one who thinks it's ironic that Microsoft did this to help out Intel, but Intel decides to repay Microsoft by refusing to upgrade its corporate machines to Vista? Wow!) My father got stuck with one of these machines from a major manufacturer and even I couldn't get Vista running well on it. The only sliver of good that would come out of this is if Microsoft really learns a lesson.

While everyone's taken Microsoft to task over Vista, part of the blame also belongs to the computer Original Equipment Manufacturers (OEMs) such as Dell, HP, Toshiba, and Lenovo. It's not like the manufacturers didn't have sufficient warning Vista was coming, but many of the problems people attribute to Microsoft are the direct fault of bad drivers for just about all devices. Add in the horrible out-of-box experience you have with your average computer that comes with Vista installed and it's no wonder Vista looks bad.

Most of you reading this are developers. What do we do when we get a new machine? Completely wipe it out and reinstall the OS. We're fortunate enough to have things like our MSDN subscriptions so we have the real OS DVDs we can use along with the technical expertise to get things working. Your average person is scared to death to do that because they don't know how to get rid of the garbage on the machine so they suffer with it. Even the machine has a backup partition where they can reinstall the OS, that image contains all the garbage the user wants to get rid of in the first place. I'm annoyed that Microsoft has allowed this to happen.

At least there's something average users can do today to avoid this mess: buy Apple hardware. I've blogged about how I put Vista x64 on my Mac Pro long before it was officially supported by Apple. Based on the traffic and comments in that entry, there are quite a few people doing the same thing because they want to control the machine and not just deal with the garbage from traditional PC manufactures.

You'll see in a second why Apple makes the best Vista machines, but first compare my experience with the Mac Pro with the hell I had to go through with my Lenovo X60 Tablet PC. As a super experienced Windows developer who has done everything from drivers out on Windows, it took me eight hours to figure out what was the minimum set of driver software I needed to use the machine. Why can't Lenovo (or Dell, or HP) give me a single install that puts on nothing but the minimum drivers necessary? Apple can do it, why can't the others? I sent a bill to Lenovo for my time getting the machine in the state that it should have been delivered. Unfortunately, I have never heard anything back. Maybe I should turn them over to a collection agency. In Lenovo's favor, they have started allowing us to uninstall some, but not all, of the garbage with their latest machines.

What makes Vista x64 so stable and wonderful on my Apple Mac Pro is that all the core drivers come from Microsoft out of the box. If you can avoid any drivers from a manufacturer and just use those in the Microsoft install, you'll have much better luck because they are so much better tested. I highly recommend never installing any drivers from Windows update that come from any manufacturer. Yes, that includes video manufacturers. Based on my experience over the years I think little or no driver testing is done outside Microsoft. If Microsoft includes it with the OS, it's been really tested and you can trust it.

So this is where we are with Vista today, but Microsoft is hard at work on Windows 7. Microsoft will finally have to deliver "Wow" when they ship Windows 7. They have to under promise and totally over deliver in order to remove the perception problems with the operating system. This is going to be hard, but if any company can do it, Microsoft can. They have great people; they just need to be turned loose.

In my opinion, there are two key things that must be at the forefront for Windows 7 development. The first is that Microsoft has to seriously crack down on the OEMs. They've tried to do this in the past with the Windows Logo program, but it's basically a joke. For the first time Windows Logo must be something with teeth that ensures that drivers and user mode software works correctly. I feel Microsoft must require that every computer manufacturer delivering Windows 7 includes the Windows 7 install disk and a single install that installs nothing but the minimum drivers necessary to make the machine work.

In order to ensure the drivers for everything are solid and workable, Microsoft has to spend some of that pile of cash they have to help anyone developing a driver to get the driver right. This could mean continuing to force more drivers into user mode or it could mean providing even more developer support. It will definitely include developing testing resources to ensure the drivers are beat to death and earn the improved Windows Logo before they can be shipped. It's obvious the device manufacturers are not doing sufficient testing, so Microsoft will have to help them do it.

In the final part of the OEM crackdown, Microsoft also must completely own the initial out-of-box experience for the operating system. Based on how bad it was when I initially booted my Lenovo, where there were actual ads when I logged in the first time, there's no way anyone will think anything favorable about Microsoft. Those initial impressions are critical, and Microsoft can't trust them to anyone else.

The other thing that Microsoft needs to do with Windows 7 is to target today's hardware, not the top of the line hardware for 2010 when it's expected to ship. The relentless focus of the whole development effort needs to be on speed. The average user should go "Wow! I installed Windows 7 on my machine and it feels so much faster now." That's what it will take to completely eliminate the "Vista's a big fat pig" perception that exists today.

Even though I make my living with Windows, I use OS X on a Mac Book Pro for basically one application: iPhoto. I've found nothing on Windows that works as well as it does. I'm not a professional photographer; I just enjoy taking pictures of my vacations and life. iPhoto does exactly what I need to make my photo management simple and easy. I've been using OS X since 10.1 so I've seen a few operating system upgrades from Apple. Each upgrade of the OS always feels faster and seems to do more with less memory. With Windows and Vista today you just know that you have to buy a new computer in order to make the operating system work as advertised. This is Microsoft's last chance to change the perception, so it's absolutely critical.

One of the hard lessons I've learned in my many years of Microsoft operating systems development is "small code is good code." At the World Wide Developer's Conference in June, 2008, Apple announced Snow Leopard, OS X 10.6. You can read more about the feature list in Snow Leopard at Roughly Drafted. What's most interesting to me is that Steve Jobs specifically said that the features are for developers and not the end user.

At first I thought the features in Snow Leopard were interesting, and then word trickled out about the shrinking size of applications under Snow Leopard. The following chart, with full credit to Roughly Drafted, really caught my attention.

Apple's doing many things to shrink down the binaries, but it's blindingly obvious that they are focusing on making everything fast and efficient. That's exactly what Microsoft has to do with Windows 7 to keep in the game. The rumors are that the same version of OS X will run on a small iPhone as well as a maxed out Mac Pro desktop. If that's the case, Microsoft needs to be very worried.

Microsoft really has their work cut out for them with Windows 7. Vista is actually quite good, but the general perception is such that they have to execute to perfection in order to eliminate the negativisms thrown their way. Microsoft has responded very well in the past when they have real competition. With Apple's OS X market share nearly 8%, it's high time for the Windows team and executive management to unleash the creative forces inside Microsoft.

I'm very curious what other developers feel about Vista and what the challenges are for Windows 7. Do you disagree with my good assessment of Vista? What do you think Microsoft needs to deliver with Windows 7? Write in the comments or your own blog entry.

So Charles Petzold tagged me for this Software Development Meme and did that hurt. Just kidding. J It's the first time I've been tagged since I was like five years old. Anyway, here goes:

How old were you when you first started programming?

26. Yes, I was a really late bloomer. In fact, that's positively decrepit in the computer world. I'm amazed they even let me use a computer at that age.

How did you get started in programming?

As my first job was Special Operations Communications Sergeant (AKA a Green Beret) the only civilian skills I had qualified me as a night watchman or a member of Tony Soprano's mob crew. Fortunately, I got hired as a manual laborer at an environmental research firm who gave me more than enough rope to hang myself. I needed to track ambient air sampling canisters I was shipping out so I wrote a big Lotus 123 macro driven spreadsheet to help. My bosses were impressed with it and kept giving more and more programming work to do.

What was your first language?

English, which I learned that as a child. Oh, you meant computer programming language, sorry. That would be SAS when I was at the environmental company doing statistical analysis for Non Methane Organic Compound (NMOC) research.

What was the first real program you wrote?

In college at NCSU, I wrote an embedded debugger for an Intel 8051 CPU for the Computer Organization and Logic 311 (Electrical Engineering for CS majors), the "death" class in the CS program. That's what sparked my interest in debuggers and solid code.

What languages have you used since you started programming?

SAS, Lotus 123, Quattro Pro, Turbo Pascal, x86 Assembler, Visual Basic, C, C++, x64 Assembler, C#, and VB .NET.

What was your first professional programming gig?

While in college, I had a full time software engineering job where I wrote a Windows 3.0 GUI application for the Center for Disease Control so AIDS researchers could share information better. It was one of the most motivating projects I ever worked on.

If you knew then what you know now, would you have started programming?

Absolutely! It sure beats getting shot at in the jungle. I feel extremely fortunate that I got started before Windows took over the world and I was able to ride that wild train. I was extremely lucky and was able to start my 32-bit development career the day Windows NT 3.1 Beta 1 shipped in 1992, which made me one of the first outside Microsoft doing NT development. I think that bet paid off.

If there is one thing you learned along the way that you would tell new developers, what would it be?

Learn the insides! If you don't know how the environment you're using works, you have zero hope of understanding how to effectively use it. By knowing how things fit together makes it much easier to figure out when things go wrong as well.

What's the most fun you've ever had ... programming?

Pretty much every time I help solve a bug that has someone stuck. I just get such a thrill out of digging in and fixing a horrific problem. That intensity and focus while you're working with others on hard problems is pure joy.

So Who's Next?

I shall tag my fellow Wintellectuals: Keith Rome, Steve Porter, Sergio Loscialo, Rik Robinson, and Andy Hopper.

Craig Shoemaker recently interviewed me about my favorite topic, debugging, of course, for Pixel 8. We had a great time on the interview and I even got to tell one of my favorite debugging war stories. We also talked about all the parts of .NET that every developer should know. Head on over to Pixel 8 and grab the download for your Zune/iPod/MP3 Player and hear me laugh a lot.

The always enjoyable Jeff Atwood wrote an excellent discussion about regular expressions that everyone should read. Having seen many times where a developer will write a full blown parser instead of a regular expression, it's critical every developer spend a little time learning regular expressions. It goes without saying that the more code you write the more bugs you have. Regular expressions aren't too hard, but like anything else us developers deal with, you do need to take the time to learn the basics so you can scale that learning curve. I wanted to chime in with a few additional thoughts on regular expression that have helped me and mention something extremely important I thought Jeff left out.

While there are references and documentation on the web about learning regular expressions, absolutely nothing tops Jeffrey E. F. Friedl's book Mastering Regular Expressions. This is a topic you need to spend some time thinking about to ensure you really have that "aha!" moment of understanding. While it's sad in today's Internet world no one seems to read books anymore, if you don't read Jeffrey's book, you'll be doing the equivalent of sky diving without a parachute. Playing with regular expressions will be fun, but eventually you will be in extreme pain.

Like learning any technology, it helps to see examples. Jeffrey's book is full of examples, but to see how others have solved many problems using regular expressions, I've always liked RegExLib.com. It's a great list of all sorts of regular expressions, many of which you can use directly. More importantly, you can see all sorts of tips and tricks. My one complaint with RegExLib.com is that they don't follow Jeff Atwood's advice and apply good white spacing and comments in the actual regular expressions.

When I got to the end of Jeff Atwood's blog post, I realized he forgot one of the most important pieces of advice about regular expressions. While you can use a web site to develop and test a regular expression, I highly recommend a standalone tool that runs on your machine. The key reason is that those tools will allow you to save the regular expression to a file so you can check that file into version control. I can't stress enough that if you're using regular expressions, the file you used to build a regular expression absolutely has to be part of the source code. With the regular expression tool file in version control, future maintenance developers, which will include you, can easily tweak and fix any problems in the expression in a controlled environment. All the standalone tools you can use support including the test data in the saved file which makes it easy for others to see what you were testing with.

There are numerous tools available but the two I find best are the free Expresso, and the commercial RegexBuddy (which Jeff highly recommends as well). If you can pay the $40.00 USD for RegexBuddy, you'll have the best tool on the market. I especially like the Debug option which makes it easy to see exactly what's matching when in an expression. In the screen shot below, I've executed one of the example regular expressions to look for the different parts of a URL. In the Debug tab, I've selected www.wintellect.com (highlighted in yellow) and the regular expression window is showing me exactly what matched in the upper window (highlighted in blue).

One thing I found a little confusing about RegexBuddy when I started using it was creating a file that contains just the regular expression and the test data as it does not use a document metaphor like nearly any other application in the world. What you'll do is click on the Library tab, and click the far left button that looks like a blank document. This creates a blank library file. Next highlight the expression from the History window you want to add, and click the Add button making sure to select "Add Regex with Test Subject" from the popup menu. The following screenshot shows you what to click. One thing I especially like about RegexBuddy's files is that they are straight XML so once it's in your version control of choice, you can compare and diff the files all you want.

While RegexBuddy is very good, but I still test my regular expressions with Expresso as well. That's because Expresso has a timer in it so you can get an idea of how fast an expression takes to resolve. This is especially nice if you're working with greedy expressions with lots of possible matching.

Don't worry, regular expressions may look hard, but with a little effort, they really are not that bad. Consider them one of the most important tools in your toolbox!

After coming back from vacation, I powered on my laptop, named BERLIN, and it took an inordinate amount of time to login. After much disk grinding, the desktop finally loaded and it wasn't my normal desktop, but a temporary desktop and Vista reported that my profile couldn't be loaded. After a very heavy sigh, I looked at the Event log and the error was "Windows cannot load the locally stored profile. Possible causes of this error included insufficient security rights or a corrupt local profile. DETAIL – The process cannot access the file because it is being used by another process."

Guessing that this error was possibly because of a corrupt file or a bad hard disk, I initiated a check disk on the reboot. No errors were reported and when I logged in again, my user account loaded correctly. Poking around nothing seemed to be remiss. All the group policies had run, I could access my servers and the internet. Now my sigh was one of relief.

Needing to copy over a few files from my desktop machine to BERLIN, I popped over to PowerShell and initiated the copy and immediately got an access denied. Switching over to BERLIN, I verified the share was there and my account had read and write access. Back on my desktop machine, I tried a NET VIEW BERLIN command and got the access denied again. Double checking with Explorer, I could see BERLIN, but accessing it always gave me an access denied error. As I was logged into both machines with the same domain account, I thought that was very strange.

Logging into my server with the domain administrator account, I ran a NET VIEW BERLIN and could see all the shares on the machine. Additionally, I could copy files to and from BERLIN just fine. Wondering if the problem was on BERLIN, I checked if I could see and access shares from it. Of course, that worked.

Figuring this might have been something with the user profile problem, I rebooted BERLIN. After it restarted, I didn't log in but tried to access the shares from my desktop machine. Again, it was access denied. After logging in with my normal user account, I started poking around on BERLIN.

Since Mark Russinovich's excellent Process Monitor shows everything happening on the machine, I fired it up and turned off all filters. From my desktop machine, I attempted to access the shares. Digging through the Process Monitor log, I didn't see anything related to my shares or the network. It was like the traffic never made it into the machine.

Now I started suspecting that the problem was on my desktop machine so I rebooted it. After logging in again, I sadly verified that the access denied was still there. Looking at the event log on my desktop, I saw the following:

That's one surreal error message. Clicking on the Event Log Online Help link took me to a page that discussed deleting the unused computer account in Active Directory. Given that nearly every single event log error you look up looks like the following, I was just happy to have a real page, even if the information it reported was worthless!

Leaving the Event Viewer open, I ran another NET VIEW BERLIN and saw the same Event 4 show up. I could run PING BERLIN and that worked, but everything reported access denied when attempting to get to the shares. I tried to use Remote Access to the machine and that reported access denied as well. Running Process Monitor on my desktop machine didn't turn up anything interesting either.

Sitting there completely out of ideas on how to proceed, I was tapping my foot furiously. Feeling something hit the top of my foot, I looked under my desk and it was my cat batting at the movement. That's when I realized she was laying on top of the network hub my laptop plugs into. Chasing her off so it didn't overheat, I found the solution to the access denied: she'd managed to unplug the network cable. Plugging that in, I had complete access to the laptop again.

Without the wired connection, I was using the wireless connection, which defaults to not allowing shares to be accessed my non administrators. The failure to load my user profile was a red herring.

Cased closed with the moral: always check the obvious first!

Looks like the WinDBG team was busy over the weekend and posted a new build of WinDBG: 6.9.3.113. The big fixes look like squashed bugs and performance improvements in symbol server portions. I bet you could drop those DLLs into your Visual Studio directories and get the benefit there as well.

You all know my geek love for Apple hardware. Yesterday Apple released the official drivers for Vista x64 support! You'll need to download and install Boot Camp Update 2.1 and you'll be set. Totally and completely excellent news!

Everyone uses Mark Russinovich's and Bryce Cogswell's excellent Sysinternals Tools. Tools like Process Explorer and Process Monitor have helped solve some of the toughest bugs I've ever worked on. I've been surprised how many people didn't know that all the tools available in a single download: the Sysinternals Suite.

Whenever I see that one of the tools is updated, I find it easiest to grab the Suite .ZIP file and unzip it to my utility directory. After having done the manual updating a couple of times, I wrote a PowerShell script to do the download and unzipping automatically. I thought others might find it useful. Note that you'll need the excellent and free 7z archive tool in your path.

As I'm still a PowerShell novice, please don't be afraid to point out anything I can do better.

##################################################################
# Get-SysInternalsSuite.ps1 - John Robbins - john@wintellect.com
#
# Note that this script requires the excellent 7Z.EXE in the
# PATH environment variable. You can get 7Z.EXE, which is free,
# at http://www.7-zip.org/.
#################################################################
param ( [string] $Extract ,
        [string] $Save )

# Always make sure all variables are defined.
Set-PSDebug -Strict
function Usage
{
    ""
    "Downloads and extracts all the tools from Sysinternals" 
   
"" 
   
"Usage: Get-SysInternalsSuite -extract <directory>" 
   
"                             [-save <directory>]" 
    "Required Parameter :"
    " -extract <directory> : The directory where the SysinternalsSuite.zip"
    "                        tools are extracted."
    "Optional Parameters :"
    " -save <directory> : Saves SysinternalsSuite.zip to the specified"
    "                     directory. If not specified, the .ZIP file "
    "                     is not saved."
    " -? : Display this usage information"
    ""
    ""
    exit 
}

function CreateDirectoryIfNeeded ( [string] $directory )
{
    if ( ! ( Test-Path $directory -type "Container" ) ) 
   

       
New-Item -type directory -Path $directory > $null
    }
}

##################################################################
# Main execution starts here.

# Check for the help request.
if ( ( $Args -eq '-?') -or ( ! $Extract ) )

   
Usage


$paramLog
= @"
Param Extract   = $Extract
Param Save      = $Save
"@
Write-Debug $paramLog

[
string]$sevenZName = "7Z.EXE"
# Verify I can find UNZIP.EXE in the path.
[string]$sevenZPath = $(Get-Command $sevenZName).Definition
if ( $sevenZPath.Length -eq 0 )
{
    Write-Error "Unable to find $sevenZName in the path."
    exit
}

# If the extract directory does not exist, create it.
CreateDirectoryIfNeeded ( $Extract )
# If there's a save directory set, us that otherwise, use the %TEMP% directory.
[Boolean]$deleteZipFile = $TRUE
[String]$downloadFile = ""
if ( $Save.Length -gt 0 )

   
CreateDirectoryIfNeeded ( $Save )
    $downloadFile = $Save
    $deleteZipFile = $FALSE
}
else

   
# Use the %TEMP% path for the user.
    $downloadFile = $env:temp
}

# Build up the full location and filename.
$downloadFile = $(Get-item $downloadFile).FullName
$downloadFile = Join-Path -path $downloadFile -childpath "SysinternalsSuite.zip" 
 
# Let the download begin!
Write-Output "Starting download of the Sysinternals Suite"
$webClient = New-Object System.Net.WebClient
$webClient.DownloadFile("http://download.sysinternals.com/Files/SysinternalsSuite.zip" ,
                        $downloadFile)
Write-Output "Sysinternals suite downloaded to $downloadFile"

# I don't like to see all the output from 7z unless there's a problem so I'll
# redirect to a temporary file and if there's any problems, I'll show it.
$temp7zOutput = [System.IO.Path]::GetTempFileName() 

# Since the -o option to 7Z.EXE cannot have a space between it and the
# directory there's a bit of a problem. PowerShell does not expand the
# line -o$Extract if passed directly on the command line.
$outputOption = "-o$Extract"
Write-Output "Extracting files into $Extract"
&$sevenZPath x -y $outputOption $downloadFile > $temp7zOutput
if ( $LASTEXITCODE -ne 0 )

   
# There was a problem extracting. 
   
Get-Content $temp7zOutput 
   
# Don't delete the download file. 
   
$deleteZipFile = $FALSE 
   
Write-Error "Error extracting the .ZIP file" 
   
Write-Error "The downloaded .ZIP file is at $downloadFile and will not be deleted."
}
# Delete the file that held the extraction output.
del $temp7zOutput
# Delete the downloaded .ZIP file if I'm supposed to.
if ( $deleteZipFile -eq $TRUE )
{
    Remove-Item $downloadFile
}

More Posts Next page »