Joe Gregorio
Some people are absolutely gushing about the latest vaporware from Microsoft. Enjoy the fantasy while it lasts because it will be a very long time before any of this sees the light of day, and even then it will be years more before it's stable enough to use in a production environment.
Let's look at this diagram of Longhorn. Focus on the lower third of the diagram, the part in white labelled "Base Operating System Services", but excluding the part in "CLR". That part, which represents about 1/3 of the whole diagram, is what Windows NT is today. That means in the next three years the size of Windows is going to triple. This does not sound like a good idea. It's taken MS years just to beat the lower third into shape, and it's still less stable and more bug ridden than linux. Now they're going to triple it in size?
Indigo
Now consider the other 2/3 of the diagram. The new stuff that is being added. On the right hand side is Indigo, a unified provider of services for security, identity and Web Services.
Taking it's lead from IBMs WebSphere - Indigo is one of those architectures that does everything for everybody. [ Marc Cantor]
Well, I'll agree with the part about following IBM's lead, but I was thinking more of SAA than WebSphere. Nuf said.
Avalon
Now let's turn our attention to Avalon.
This is a whole new layer of Windowing and Media services layered
on top of .NET, which itself is a layer on top just like COM, which is
layered on top of the Win32 API.
At each point in the evolution of Windows, each layer was supposed
to be "the cat's meow" and end all of our troubles and enable
"rich new interactions". First note that none of the new
layers has ever had that effect. Second, ask yourself, what does
this whole new layer add? To quote InfoWorld:
Emphasis mine. Why does it look the same? Because all the changes are aimed at the developers. Ole Eichhorn say's it best in his essay:
The most important thing is not how easy it is to build code, the most important thing is how well the code runs once it is built.
Yet again another new set of interfaces to learn for the programmers, which this time we really really promise will make you more productive. And even if it doesn't make you more productive, you'll have to use it anyway since you won't have a choice:
Longhorn will be the first operating system where ALL functionality is designed to be accessed through managed code. WIN32's reign as the Windows API has ended, replaced by managed .NET APIs.
Such a move is unique in the industry. No one, including Sun Microsystems, has made Java THE API for programming a particular OS, much less declared that, going forward, all new features would be offered through Java. Microsoft's decision to do that with .NET is an important advance, as it FORCES developers to write more secure code, simply because they can't make some of the coding errors (such as buffer overruns) that form the lion's share of security flaws.
Remember the last time we heard such breathless copy over a Microsoft technology? Maybe the same promises were made about COM? At the time they were already getting a little stale because they'd already been used in hyping MFC.
The last thing to ask when pondering Avalon is how many applications has Microsoft shipped that are pure .Net applications. After all, that's what they're telling their developers to do, and that's all of what Avalon is built upon. Last I knew, that count was still sitting at zero.
WinFS
WinFS is the file system formerly called Cairo and has repeatedly not shipped since 1995. If it ever did ship it would be a complete failure because it does not solve a problem that anyone actually has. The whole idea revolves around attaching meta-data to all your files. Does this sound familiar? It should, and all of the same arguments about user supplied meta-data apply, see Doctorow, Shirky, and Pilgrim.
Now assume (incorrectly) that you were real careful and went back and entered meta-data for the 20GB of your special files. You know the ones, the same files that you have not been annotating with meta-data over the past 5 years. What are you going to use WinFS for? Why to search for pictures of granny or for old documents. That's it. Just search. Why all the fuss over meta-data? What you really want is Google for the desktop. All the meta-data in the world isn't going to find the mis-filed final revision of your marriage proposal titled "Untitled7.doc" sitting in "c:\Documents and Settings\Owner\Application Data\Microsoft\Office", yet a simple text search for "poem love marriage" will likely turn up all 7 revisions.
That basically covers it. Three major components, all useless in their own right, now stacked together on a shaky foundation to make an even more useless heap. Like I said at the beginning, don't expect to see any of this make it out the door and into a production environment for many many years, if ever.
Update: First I would like to clarify some things.
- This was not directed at Microsoft the company, nor at the employees of same. I have only commented on the technologies themselves. I was suprised at some close-to-ad-hominem responses I saw earlier today, and I am pleased that they have since been edited to stick to just the technology arguments. I have, for all but a few exceptions, been impressed with the intelligence and character of Microsoft employees. My feelings for the monopoly of Microsoft the company are well documented elsewhere.
- I am rather impressed with the amount of feedback directly from Microsoft employees working on parts of this technology. That kind of public interaction would have been unthinkable just a couple years ago.
- I have corrected a mis-statement above about .NET being a layer on top of COM.
- I never would have written this post if I hadn't come across, as Dave Johnson put it Marc Cantor's "gushing fanboy Longhorn post".
Mike Deem has a rather lengthy rebuttal which covers some good stuff. I believe most of the things he talks about are "solved" by having a search engine running over your files. The only thing I want to point out in particular is this:
But, WinFS does not stop there. When you put a file in WinFS, a property handler that knows the internal format of the file is executed to pull selected data out of the file to make it searchable.
My only response to that is, yeah, those proprietary file formats are a pain, aren't they?
I disagree with your claim that the functionality they plan to provide is useless. This is the way I see things
Avalon: New Widgets for Windows GUIs
Indig The next generation of DCOM but supports XML Web Services so you can interop with anyone with an XML parser
WinFS: A standard API for tagging files with metadata and perform searches on the file system.
All of these are useful. Now you could argue that since this is Microsoft then it will all suck until version 3.0 (or whatever the cliche is) but saying they are all useless makes you look quite narrow minded as a developer.
Posted by RE: Longhorn versus the light of day on 2003-11-08
Ooh, what a burn. I disagree with your claim that the functionality they plan to provide is useless. This is the way I see things
Avalon: New Widgets for Windows GUIs
Indigo: The next generation of DCOM but supports XML Web Services so you can interop with anyone with an XML parser
WinFS: A standard API for tagging files with metadata and perform searches on the file system.
All of these are useful. Now you could argue that since this is Microsoft then it will all suck until version 3.0 (or whatever the cliche is) but saying they are all useless makes you look quite narrow minded as a developer.
PS: The fact that you specced the CommentAPI yet your site is the only one RSS Bandit has problems posting to tends to the ironic.
Posted by Dare Obasanjo on 2003-11-08
Joe, were you at PDC? If you weren't, I highly suggest you read a bunch more before jumping to these conclusions. If you were, then I guess our messaging wasn't as clear as I thought it was.
Avalon is not just a windowing technology that looks "alot like current versions of windows". You reference Infoworld here; I wouldn't invest too much in that opinion. They haven't written code to it, and they haven't used it. Like everyone else, they saw about 15 minutes of video. Avalon/Aero is much more than most people think right now.
Indigo is not the next DCOM, or the next WS*, or the next COM. Its the new way that applications interoperate. yes, the scope is huge. Years of research, feedback, and expertise are going into its design. The community is being consulted. If theres something you don't like it its design, speak up! but be specific...
And winFS isn't designed to help you organize 200 gigs of your useless files. Its designed to handle 2 terrabytes of your useless files. It doesn't just let users append poor metadata to their files; it provides a framework for users and applications to tag metadata, in ways relevent to THEM. beyond that, it doesn't just let you tag your items, it (in my opinion, more importantly) lets you define the complex relationships between them. Yes, it all sounds very semantic. But its all about MY semantics. google is text. WinFS is much more than that.
I've rambled on enough. This really should have been an entry in my blog. sorry if i'm wasting your bits. :)
Posted by Chris Hollander on 2003-11-08
Wow, Chris, that's so completely the point Joe was making. If WinFS requires users to do any work at all to tag their own data, it's completely useless. Well-intentioned architecture astronauts have given users all sorts of tagging opportunities for years, and it's all completely pointless, because nobody uses it. They don't use it even when it's in their own best interests to use it, and if they are forced use it, they get it wrong.
Just for fun, have you searched for a "plam pilot" on eBay recently? Yeah I know it's an old joke, but damn it, I just searched and found one. In fact, in the past several years, I have periodically repeated this search, and I have never not found one. You can always get a great deal on a Plam Pilot on eBay.
See also: http://diveintomark.org/archives/2003/04/08/the_importance_of_titles
Posted by Mark on 2003-11-08
But couldn't the OS and applications tag files with useful information without any additional effort required from the user? The negative comments here seem to be focusing on the Property Editing features of WinFS, which may never be used. see http://msdn.microsoft.com/Longhorn/understanding/ux/default.aspx?pull=/library/en-us/dnaero/html/wux_topic_storage.asp
Seems like there's a lot of room for application improvement without any user data entry.
Some examples:
1.) Quark Xpress could establish "holding relationships" for each document's linked image files and fonts. It wouldn't matter where the files lived. Eliminates the need for programs such as Flight Check.
2.) An email comes in from a developer mailiing list concerning an API change. The email application could list source files the depend on the API, or the IDE could display recent correspondence on the matter.
So far, this discussion has also assumed that Windows is pretty good at what it does today. It really isn't. Opening a folder with a large number of items and displaying their application-specific icons can be slow, even on a fast computer.
Posted by Robert Sayre on 2003-11-09
Mark, that's exactly why Google is so popular. It derives all the metadata it needs from the web sites themselves without the web authors needing to do any work. If it relied on meta tags to do its PageRanking or indexing, the spammers would bring it to its knees.
From what I've read, WinFS seems like BeOS's system but I don't know enough about either to fully grok the differences.
Posted by Bill Brown on 2003-11-09
Posted by Ziv Caspi on 2003-11-09
Ziv, a "personal Google" (Google on the desktop, as Joe put it) would have to do relevancy ranking differently than Google-at-large. Google-at-large relies (somewhat) on the linking structure of the web. As you point out, there is no direct analogy on the desktop. But there are usage patterns, and I suspect any program that could track them properly could rank the best results of a full text search. But again, that's the thing -- people search for text, and they don't tag their own metadata, so full text search + smart automatic relevancy ranking is really the best you can do. Luckily, that's enough (as Google-at-large has shown us).
Of course, this only works for text, or things that can be treated as text. There's open source software that can burrow into Word and PDF documents and extract text; images would be much harder since there's not enough to go on. Directory name, filename, maybe some dates in the EXIF data if you're lucky. Google-at-large's Image Search relies heavily on ALT text and text surrounding the image; images on your desktop don't have as much of that kind of context.
Anyway, I'm rambling. Metadata: good. Metadata: hard. Ergo, metadata-based search: useless for end users.
Posted by Mark on 2003-11-09
While it is true that there is no linking among documents today, that in no way dooms "Google for the desktop". The difference in the scale and focus of the documents you are indexing. Keyword search on the scant amount of documents I produce works just fine. That is, scant in terms of the volume that Google indexes.
And if I had to bet between users adding meta-data manually to documents, or using simple linking technologies, I will always vote for the solution that requires the least energy.
Posted by Joe on 2003-11-09
Posted by Mike Deem on 2003-11-09
Posted by Randy H. on 2003-11-09
The obvious issue is how much metadata can be extracted automatically. Sometimes it's very little. Other times, quite a lot.
I wonder how many people actually use the Indexing Service in Windows, for example -- many people in Microsoft use this tool to index all the source files on their machines, so they get sub-second search for all identifiers in their programs.
Posted by Ziv Caspi on 2003-11-09
a couple of quick comments, in regards to "making" users tag their content:
first, it wouldn't just be users tagging content, it would be their applications, and their OS tagging as well. there is tons of metadata that could be harvested, and currently isn't lets start with:
"what documents are often opened at the same time as this one?"
"what documents did i read within 10 minutes of reading this one?"
"sort documents by average editing time?"
"who have i sent this document to?"
these are all dead simple examples, and even they could add tons of value. combine that with all of the other simple ones, combine that with more complex ones (location, etc), and combine THAT with a slight burden on the user to add juuuuust a tiny bit of really, really relevant metadata, and you have tons of facts that you can base some pretty interesting inferences on.
Second, a large contingent of users already tag their content though folder structures. the problem, of course, is that users are "locked in" to a single organizational structure; once i figure out my folder structure, my files can't exist in alternate structures as well. they should be able to, though. For example; my wife is completing her MSW. she's organized her various papers into folders by class. she might also want to organize them by year, or by references, or by whom she collaborated on for group papers, or by professor, or by grade. she can't do that now. that sucks. even more, she can't "mix in" emails that she's gotten, or bookmarks from her browser, or relevent IM contacts. she wants to, though, and gets frustrated that she has to switch from app to app.
again, I've written more than I wanted to, but I have one last comment to make; Mark, I find it ironic that you've used a syllogism to describe how metadata doesn't work. ;)
Posted by Chris Hollander on 2003-11-09
Good stuff to think about. I'm actually experimenting with an alternate approach: dump everything into HTML and index the lot. That is, run LXR over my source code, dump my inbox into HTML with Hypermail, expose my Moz calendar info in HTML, etc. Then use a search engine like Swish to search over the whole set of pages.
Posted by joe on 2003-11-10
Randy: what you describe is already present. Have you ever looked at the properties on a Microsoft Word document? There's title, keywords, categories, and so on and so on.
How many Word documents have you added such metadata to thusly? How many average users know about this feature or how to use it? Even less actually do it and still less know how to use it to find documents.
It's built right into the app, not out there in the OS to manage metadata. I think that Microsoft will probably have to make an external metadata manager of some sort because they don't control every app out there and the utility of metadata comes into play when it's pervasive.
That said, I'm sure that they'll be featuring metadata more in their own apps. But how do you get around the education issue? You know, this is why metadata is important, this is why you should take the time to think through your metadata system, or this is why you should use the one we've set up for you. Do you make them enter the metadata prior to the first save? Probably not since that's an inconvenience. How do you enforce it?
Posted by Bill Brown on 2003-11-10
"The last thing to ask when pondering Avalog is how many applications has Microsoft shipped that are pure .Net applications. Last I knew, that count was still sitting at zero. "
Sorry to tell you, but MS Reporting services due next month is written with .NET code. Written with C#, and obfuscated.
Posted by Stephane Rodriguez on 2003-11-10
Thanks for answering my question with, as far as I can tell, a positive answer, that is, that MS has not yet shipped a .NET application. Yet inspite of that dearth of experience, it is willing to invest in re-designing the whole user-interface construction mechanism of Windows on top of a foundation of .NET.
Posted by Joe on 2003-11-10
Joe: "...MS has not yet shipped a .NET application. Yet inspite of that dearth of experience..."
Some shpping applications do use .NET technology (VS.NET, for example). In addition, a large number of internal applications rely on it. My group, for example, has re-written its entire test suite in .NET, with good results.
The fact that there are no shipping products which are "mostly .NET" does not mean we have no experience with the technology.
Posted by Ziv Caspi on 2003-11-11
Several comments:
1- Chris, to organize files redundantly in a folder hierarchy, consider symlinks (or the badly adapted windows shortcuts); to insert bookmarks from IE, simply drop them into your folder. No need for an extra complicated WinFS to do that.
2- Want widespread use of metadata? First stop calling it metadata. Call it something more human. My parents understand the word "folder". They don't understand metadata. They don't want to know. Call it index, keywords, extra information, whatever looks natural and gentle.
3- Users already add metadata to their files. At least one kind: look at at your mp3 collection. ID3 tags make song indexing so convenient that either softwares will automatically generate the appropriate tags or users will go tagging their old files.
Sure Office Word can add keywords. But it's hidden in a subdialog somewhere. Make it easier by providing entry fields in the save dialog, and make it obvious to search for tagged documents and users will get used to it if it serves them. For most documents, the name contains the keywords anyway.
4- Different users have different ways of thinking. Several BeOS users I knew relied a lot on the index/search built in the OS. I never liked it and I did develop on the BeOS for a long time - I remember where I store things. Some don't (the typical "where's my file?"). So in this regard, WinFS is a good thing. Eventually everyone will use what suits her/him most.
Posted by Alfray on 2003-12-08
I'm with Alfray on point #3 there (MP3s, etc)...as soon as I saw the ID3 tags, I went crazy. It made organization hella easy, since my P2P client was already set up to organize based on ID3.
My question is this (maybe it's a naive one): how much HDD space are we sacrificing to MetaData?
Posted by Erick on 2003-12-23
The capability of the WinFS product to actually get released is NOT something I will address.
The concept and technology of what WinFS intends to become is worth discussing.
WinFS contains exactly ZERO lines of code from Cairo, it's foundation is Microsoft SQL. SQL is not even mentioned on this page. File promotion is a backward compatibility feature but not the heart of WinFS.
No user need worry concern themselves about how Outlook and PowerPoint (as a winfs aware application) shred their data into the same file system. However new and unthought of scenarios will emerge that take the PC experience to the next level.
Read Ray Ozzie's comments and see the big picture. At the moment you fail to see the whole picture.
http://www.ozzie.net/blog/stories/2003/11/14/640kbOughtToBeEnoughForAnyone.html
Questions can be sent to ms_outsource@hotmail.com
Posted by WinFS engineer on 2004-01-16
Posted by WinFS engineer on 2004-01-16
Posted by ben on 2004-11-04
If MS is really taking so long to do thing, I'd assume they'd be layer any Win32 APIs on top of the .NET interfaces, not the other way around. Either way, it's going to be hard to get decent performance out of this, but by they, PC users will all have 4 GHz processors and 4 GB of RAM, right ?
The COM interfaces can be exposed by .NET objects (this is already supported), so that is not so much of an issue.
Posted by Bob on 2003-11-08