Delphi, Uses, IFilter, and MSN Desktop Search

It has been ages since I have posted anything and I have not gotten around to posting on some of the items I had in the queue but that will happen eventually.

I am sure many people are aware that the desktop search choices are coming out fast and furious. I have about 500K files/items on my local computer and I am definitely in the habit of misplacing a particular file often enough.

I hadn't bothered with the first Google Desktop Search (GDS) as it would not index content outside of a pretty refined set of file types. The recently released version corrects that problem and is a pretty small download. It does a much better job in what it will index and is extensible with some programming (using IDispatch based interfaces, tsk tsk). I am sure it will be a decent tool but I don't really have a high degree of trust in letting it into my Outlook data just yet. I have it installed on a Virtual PC to give it a test and play around with it. Since I use google as my mainstay web search it isn't a stretch to doing it on my desktop as well.

I have had the MSN Desktop Search beta installed since it was released. I simply couldn't pass on an opportunity to quickly search through my Outlook e-mail. The packrat in me has led me to having an archive policy set to six months and over the years I am up around 3GB of archived e-mail. Searching into those archives in a reasonable fashion is a dream come true. MSN Desktop Search does it both quickly and reliably. Searches against Outlook are incredibly fast and it hasn't let me down on a single search yet.

A secondary, although more often used, search need is grepping through the volumes of the source code (predominately Delphi). Typically I have stuck with the dos based grep and/or the Delphi editor supplied version. This works for many cases but the search speed certainly doesn't compare to what you find from either of the two search engines I have tried thus far. For code and html documentation libaries you simply can't lose by installing one of these search engines.

But... I want more. The MSN Desktop Search takes advantage of the IFilter interface that was designed for other Microsoft Search technologies such as Index Server and SharePoint (I am not convinced that the new search is anything other then a new face on an already pre-existing and robust technology). Using the IFilter interface permits you to use any existing filters that may exist such as the PDF filter that is linked on the beta download page. But, more importantly, it allows the addition of new filters that you can develop yourself for custom formats or specialized needs.

This was my first step into filter development so I started with something simple that would be of potential benefit to myself. I have many shared modules between many projects so a change in a particular module can have far reaching impacts. These projects can be old mothballed projects, up and coming work, and large efforts with many external dependencies that I need to watch for. Before, or more likely while, making some changes it is definitely an asset to initiate a light impact analysis to see any dependencies that should be watched for. So, to this end I figured a uses filter extension would be just up my alley. Since it involves parsing the Delphi source it also permits extending the search patterns to include searching for where classes and functions are used versus defined. Part of this is about just having some fun doing it versus building up an impact tool... good for helping with first guess estimates.

The uses filter parses Delphi source files, establishing the list of used units. Additional custom properties are added with the list as well as setting the standard keywords property with that list. This means I can execute a search such as "keywords:unit_name" and I am then presented with a list of Delphi source files that use that particular unit. This helps keep the search results short and usable since the words are not scanned in every e-mail, document, and text file floating around your local machine.

The picture belows shows a search that shows modules using two units at the same time. You will note that the named units are in the keywords properties (this also means we don't find units where the unit may have been historically used and may have been commented out, etc).



Delphi, of course, is the tool I used. After stumbling around for a while I ended up building up a filter interface file with the relevant declarations for constants, types, and interfaces. I then created a new ActiveX project and created the basic framework code with support for the necessary interfaces and stub implementations. Be an appropriately lazy developer, I decided to implement the filter as an extension/aggregate of the stock text filter the desktop search was using and simply tag my extra bits on afterwards. This means I wasn't losing any basic text search capabilities either. To this end, I was able to get the filter up and running pretty quickly with the help of the filter test tools from the MS SDK. Using Martin Waldenburg's excellent mwPasLex source code made accurately extracting the uses from the source files a breeze. After setting a few registry entries (see the MSDN IFilter for details) I was up and running with a working copy pretty quickly aside from the long re-index times required for good testing. The moral of that story is don't test on a machine with 500K files; thank heavens for virtual PC.


I am still testing and tweaking the code otherwise I would make it available on the site. If you happen to be interested in having the code (for any purpose) just leave a comment with a suitably spam-proofed e-mail and I will forward along the current version. I want to figure out how I can get the desktop search to respect my new properties for both display and searching. I didn't want to use the keywords property but all my initial attempts to get the search engine to use my "uses:" prefix simply failed. If someone knows how that works (not on SharePoint or Index Server) I would be quite interested since I think is where the real power resides.