Fog Creek Software
Discussion Board




"LongHorn" - Nest generation Windows OS

hi,
have u happened to see this link ..
  http://www.winsupersite.com/faq/longhorn.asp

do u think is it revolutionary?

MRC
Friday, December 06, 2002

Not at the moment but it could be. Or it could turn into another "Cairo" project for Microsoft.

John Topley
Friday, December 06, 2002

All depends on how the final release looks and does?

Prakash S
Friday, December 06, 2002

Be sure to look at Thurrott's review of the alpha version (not the "Road to Longhorn" review).  It has more recent screen shots.

http://www.winsupersite.com/reviews/longhorn_alpha.asp

Joe Paradise
Friday, December 06, 2002

I think the most revolutionary thing about Longhorn and other future windows releases is the SQL server based file system. Can you imagine the reportng/data viewing capabilities this thing will have? I'm looking forward to that.

What I'm not looking forward to is digital rights management, palladium based hardware, and other Hollywood inspired garbage. That type of technology is just begging to be abused.

Ian Stallings
Saturday, December 07, 2002

Hmm SQL Server as a file system.

So I guess all the times to populate list boxes of files are going to take even longer now.

Select * from filesystem where splinge-oops-this-string-buffer-was too-short

Simon Lucy
Sunday, December 08, 2002

DRM stands more for Digital _Restriction_ Management than it does for Digital _Right_ Management. (Equally well, Copy _Protection_ is better labeled Copy _Restriction_.)

A better term does wonders to other people's perception, especially if they are not versed in technology.

Ploni Almoni
Sunday, December 08, 2002

I have worked a lot on systems that include a database engine as part of the os. In fact, this means that simple dir command would in fact actually create a query and submit it to the database engine. Thus, virtually everything in the system was stored as a database record. That meant that each entry, and even each command (such as "dir" and copy)  for the os was actually a entry in the database. It was great, since then you could ask the system to show a available list of commands! You could also easily change the language of the system, and localize it. You could easily go:

Copy dir to:
MyDir

At that point, you would have two commands that do the same thing (this was a command line driven system).

During code development, you could use query commands to select groups of code modules, and then copy that "set" or selection of modules to a target directory (you could also easily save your selections for later use). Hence, you could even say only copy files that DO only exist in the target directory. This was Really cool. I am of course talking about the Pick system. Today, generally the system is NOT the native os anymore. It used to be, but now runs on NT, and runs on top Linux. However, for a good many years it was also the native os, and that OS was JUST database engine. The fact that the system was also a developed around a pcode system (like java) is probably why it still exists today. However, it was also widely used commercially 25 years ago too!. If you want some links, or want to learn more about this multi-valued system. You can read my article at:

http://www.attcanada.net/~kallal.msn/Articles/fog0000000006.html
 

Ok, there are two very distinct issues that you get when you include a database engine as a integrated part of the OS. It is critical to understand the two issues.

Issue #1
As mentioned above, everything is stored in a database. That brings up all kinds of new ways to think about how you mange your files. The example of copying a bunch of files to a target dir for files that ONLY exist in the target dir is a good example (how does one do that windows today? anyone?).

It also of course means that searching, and organizing files can be a very different process. It also means in general you can extend the file system. So, in general you have standard  Date, Time, Filename etc. fields that you get with all directories. However, you now can also add additional columns of your choice!! For example, you might add Last network station that used the file. You might add a comments field that allows you to enter a bunch of keywords to later help you find the file. In a legal office, or medical center, this is a powerful idea. You now have a high speed system wide searching for key words, but not have worry about if the document is in fact a digital picture (fab feature of  a database driven system). Hence, you add all kinds of stuff and extensions to the file system.  In fact, I found this feature of the OS often abused as some applications would start adding all kinds of junk to the dir database! Do you really want software to start modifying the dir colums?

It is also much easier to code stuff that reads directories since any code simply treated the dir's as a database. It makes life great for programmers.

Albert D. Kallal
Monday, December 09, 2002

Issuse#2
This is the real brass ring bonus I am talking about. However, none of this advantage I am talking about will be realized until vendors take advantage of this fact. This is a classic chicken and egg program. None of what I am about to talk about will be realized on day one, but down road it can mean everything. Thus, this advantage will be realized when vendors actually adopt longhorn.

What I am talking about here is that fact that now the OS has a built in database. It means that every single application out there can use a common data format. The most useful thing about the Pick system (a database os) was that if you purchased a XYZ accounting package, and then went out and purchased a ABC CRM (customer relations package),  is they both stored their data in the database!. That means I could read, and query and in fact use the data without even launching the respective application. I always loved the pick system, since any data in the system could be looked at, and used WITHOUT even opening, or using the vendors software.

In windows this would mean that you could send me a quicken accounting file and I would be able to read it. Same goes if I installed WinFax software, the SOURCE for names (and fax numbers) used for the faxing program could come from ANY other program (after all, we have a common data format, and this thus just be a query). This of course assumes that every single vendor used the database engine (in place of their own proprietary format).

This is the biggest advantage of having a database engine as part of the os!

I also think that vendors would in fact jump on this (they did with pick!!). It would become instantly obvious that each application should store their data in this system database. This is a fabulous way to make software integration real easy. Once you learn that database system, each new program that comes along will now use this format. This is VERY powerful concept from market place point of view.

For example, the Simply Accounting package for some time has used the JET (ms-access mdb files) database format. This just means that virtually any computer running office can actually import, and even modify data directly in the database (since it uses the same format as ms-access). Well, sure enough, there is a good activity in small VARs who now customize Simply Accounting for small business. These people also now integrate other software systems to work with Simply Accounting. These vars can do this, since Simply Accounting used a very widely used database format (JET - mdb). Also, ms-office has all kinds of tools to pull data from JET files, and thus ms-office integrates BETTER then if they choose their own format (such as something like dbase format).

In fact, Joel's CityDesk web content management system also uses the JET engine (mdb format).Joel had to use something ..right? That means you can in fact open up Joel's files with ms-access and fool around. The bottom line here is that if the OS has a widely used database engine, then vendors will use it, since it solves all kinds problems. One real big problem is now you DO NOT have to choose, or setup, or even install a database engine anymore!!. Every program and it's dog would jump on this fact (this why companies would use this!). (Joel picked JET since it was there!!...Simply Accounting picked JET since it allowed more integration then their competitors).

Regardless, I think everyone can now understand the ramifications of every single vendor using a common database format to store their data. You can now look at any vendors data. In fact, if the program is removed, or the vendor goes bankrupt, you still have easy access to the data. That is also probably why pick is still used to day. (no data format lock in is possible!!).

It always struck me as real stupid that how each vendor used to have to write and maintain their own list of printer drivers (pre windows days). It also struck me as very dumb as to how each vendor has to now choose what, and how their data for the application is going to be stored. 

Unfortunately, this 2nd most powerful benefit of a database driven OS will not be realized until each vendor drops their own internal format used for database, and place their data in a common format.

Note that for the pick system there was in fact MORE database development tools written as result of a common engine. In other words, if FileMaker, ms-access, dbase , and Oracle were installed on your pc, they all might be very different products, but the all would store their data in a common format.

In other words, the fact of a common database engine does not hinder, or hurt the fact that vendors want to create database/application development systems. In fact, Pick was so good, that I even wrote my own development system (and there are still people today using my tools to develop software around the world today). I was able to do this, since the engine was there, I just had to build a forms package, and a report generator.

Hence, the GUI, the printer drivers, the sound drivers and a lot more was once written by each vender.

It is high time, that the database format is not re-invented each time. We need a standard database format *AND* engine for the os to really open things up.

Albert D. Kallal
Edmonton, Alberta Canada
Kallal@msn.com

Albert D. Kallal
Monday, December 09, 2002

Albert,

excellent thoughts. Where do you think XML fits in?

My guess is all the data interchange between any application to the database will be in this format. The application might be MSFT's or some third party app, but interchange would take place in this way.

The cool thing is you could do it over the web to your desktop as well.

AM I missing something, other than this?

Prakash S
Monday, December 09, 2002

Pick, though, has largely become a dead end.  That might be regrettable but true.  In the end perhaps what helped to kill it was the completely enclosed system that applications had to live in.  (I know there were ways to break out of the box). 

It made development more expensive even if, as claimed by Pick houses, they could develop robust applications faster, because there were fewer developers around that could use it, there was a learning curve to get into it.

In the end it became a niche and niches are always out evolved.

Oh, and I'd probably trust Pick as a file system because it was developed that way, a database file system developed by MS gives me the heeby geebies. 

Though I've already decided that Win2K is the last MS operating system I'll work with voluntarily anyhow.  Just call me a dinosaur.

Simon Lucy
Tuesday, December 10, 2002

dinosaur:-)

Prakash S
Tuesday, December 10, 2002

Simon Lucy, what have you done. Now I am going to be stuck with an image in my head of poor little Pick apps stuck inside a brown cardboard box, banging on the sides yelling in thin, child like voices, "Let me out! Let me out! I won't be bad, I promise!"


Tuesday, December 10, 2002

So, having very nearly got caned for integrating their browser into their OS, the same company is now integrating their database into their OS? Huh?


Tuesday, December 10, 2002

To anyone but a developer, a file system with disks, directories and file names has no natural meaning, so a storage system that is more like a database as far as retrieval is concerned is probably a very good thing.
Provided it is designed properly, taking into account both technical issues, as well as usage issues. Meaning, for instance, increased flexibility should not be voided by decreased performance.

If Microsoft can provide such a feature and build it with their database technology, that in itself would not be a bad thing. But if the technology is extended to the user (being an application developer or a an application user), thereby more or less nulifying the need for competing database technology, then it would be a bad thing.

That is how it would be different from the browser thing. The browser being an end-user tool, with it integrated into the OS, it is hard to find compelling reasons to acquire a competing browser.
With database technology integrated into the OS to support some user feature, and to do no more than that, there is still a market for generic databases.

Erik
Tuesday, December 10, 2002

You know, history always repeats itself.

The BeOS people tried the "Database as file system" idea back in the early 90's, and abandoned it because they couldn't get decent performance out of it.

They did, however, come up with a really cool file system with extensible metadata for every file. If we could just get that under Windows, it'd be a big, big win even without the full db.

Chris Tavares
Tuesday, December 10, 2002

>>You know, history always repeats itself.

>> The BeOS people tried….


Well, pick was alive and well in the late 70’s and early 80’s.  Back then the idea of a powerful computer was putting 10 users on a 68000 box with 512k of memory. (and those kinds of systems ran very well indeed). I have seen pictures of a 386 pc version of pick that was rated for 17 users. Pick is known for using almost no memory.

In addition, each record in pick is stored as a string. That exactly what XML does. Ie: one record can be a invoice like:

Name
Address
Phone

Product  Price Description qty etc.


In other words, some fields like name and address are for one value, but the fields for Product, Price qry etc can have a depth (is in fact a table). However, the whole thing has to be a single record for xml to work (that is why XML is better then dbase, or a simple csv text format). (dbase, or a csv file can only define a set of values that repeat. You cannot define a single record in dBase to represent a invoice for example. You need two tables:

1) the simple customer fields
2) The repeating values for the invoice details.


Systems like pick and XML allow one TEXT record to represent the whole invoice. The fact of TEXT here is also important. Pick did this from day one (all data is text). So, why back then they figured out how to get good performance with a string based database?. You wonder what they are doing wrong today with the claim that XML cannot be a high performance database. Further, the pick system was, and still is a p-code system (gee, is anything new today?).

>>Pick, though, has largely become a dead end. That might be regrettable

Yes, just because some cool system out there exists is not a big deal. In fact it means very little. (beta Vs VHS…we all know the story). However, Pick was the first commercial database company to adopt Linux (and as a result, it is not a closed system anymore). They were the first commercial database vendor, and I believe were at the first year of LinuxWorld.  This is several years before companies like Oracle or IBM had even heard of Linux.

IBM also does have a line of multi-valued systems (pick compatible systems). They got those systems when they purchased Informix. The home page for IBM’s multi-value systems can be found at:

http://www-3.ibm.com/software/data/u2/

So, the use of pick, and its variants is not exactly dead right now, but the future does not look the best.

Of course the other popular vendors are:

http://www.jbase.com/  (a pick compatible system with a pick to C translator)

http://www.rainingdata.com (the original flavor).

http://www.revelation.com/ (they seem to be showing new signs of life!!)


Anyway, I had wanted to mention more on XML, but my post again is already too long!!!

While xml may be a common data format, right now most XML is accomplish via COM, DDE, or some type of code/program interface.

That is not the real ticket we get with a common database engine. If quicken, ms-access, and CityDesk all decide to use xml, I still cannot query those packages via some “general” data engine (at least now I can not!!).

XML is a standard data exchange, but has NOT yet become a standard file format. It is unlikely we will see quicken store its data as 100% as xml. So, right now Quicken reads it data, and * then * sends out the xml (if for example Quicken was to use XML, or be enabled as a web service).

I should be able to delete the quicken accounting package, and still read/query the data files. That is what a true data standard really would bring.

A true data standard means that the data structures of data is outside of the program.

Further, the current database engines (with the exception of Pick) are NOT suited to map a relational model to a xml model very well at all. The only workable solution right now is to convert sql to xml after a query is done.

I also surprised at comments about a database engine for directory being slow.

In fact, directory services from Microsoft was based on a JET variant for years..and still is may be now for all I know…

A database engine is MUCH faster then the standard directory stuff we have now. 100,000 entries in jet can be * easily * searched, and even sorted in well under 1 second. Searching, and grabbing of data is MUCH quicker in a database then just a large file system.  Especially when you can give the data engine the memory and processing it needs. If you cannot give the engine the memory, then of course the data engine is a bad idea. One might argue that modem dir systems in windows and Linux already are a database system anyway.

At the end of the day, it is unlikely we will see this concept of a common data format * and * a common data engine for our data. (we need both to make this work!!). It is a nice thought on my part..but not something I am not holding my breath for…

The next version of ms-office does look like it will use XML for storage..so who knows, the future just might finally get this right…


Albert D. Kallal
Edmonton, Alberta Canada
Kallal@msn.com

Albert D. Kallal
Tuesday, December 10, 2002

Albert, different Pick software vendors used the same data format because at the time, software development was a professional activity, and price and performance (quality included) were the two deciding factors for the client. Also, the landgrab mentality was virtually nonexistant (it appeared in the mid 80s, I think).

Most modern apps I've had the pleasure (?) to work with can use either Oracle, SQLServer or other databases, and there are usually bridges for products that don't natively support a database - so that a client can, in theory use the same database for all of his CRM / Accounting / etc. needs.

But every vendor uses a _different_ schema, not documented but usually changing between versions; This makes it nearly impossible for software from different vendors to co-operate unless they strike a strategic deal (and often, even after they do strike one). Which is something most vendors consider good for _their_ product, because it promotes vendor lock in.

The market today is not driven by needs, performance or quality - but rather by marketing, FUD, and politics. well, it probably always has been to some extent, but the recent 5 years have been the worse yet (and getting worse all the time)

The availability of a database won't promote interoperability (ODBC/OLEDB could have done that as well, but didn't, and the reasons it didn't aren't going away). If you take a sober look at Longhorn or .NET, you'll notice that the only real problem it solves are for Microsoft, not for you (unless, of course, you ARE microsoft .....)

Ori Berger
Wednesday, December 11, 2002

"If you take a sober look at Longhorn or .NET, you'll notice that the only real problem it solves are for Microsoft, not for you (unless, of course, you ARE microsoft .....)"

Can you quantify that statement?

John Topley
Wednesday, December 11, 2002

Chris Tavares:
"The BeOS people tried the "Database as file system" idea back in the early 90's...they did, however, come up with a really cool file system with extensible metadata for every file. If we could just get that under Windows, it'd be a big, big win even without the full db."

NTFS can already do this. It has had extensible attributes since day one. If you look at the properties for files on NTFS volumes using Windows 2000 or Windows XP, they have custom properties that you can fill in and perform searches against. I believe these are stored in a separate NTFS stream.

John Topley
Wednesday, December 11, 2002

Albert,

I think the efficiency question with the BeOS file system came up because they were doing things like storing movies and mp3 files, not invoice records.

How does the Pick DB do when you've got multi-hundreds-of-MB blobs?

Chris Tavares
Wednesday, December 11, 2002

NTFS does extensible metadata?

I think I've heard of this, but of course nobody uses it because Win32 software is still effective stuck in the days of FAT. Reparse points are really cool too, but nobody uses them either.

<Sigh>.

Chris Tavares
Wednesday, December 11, 2002

<< Can you quantify that statement?  >>

No, I don't think I can in any meaningful way, but neither can the converse be quantified, I think.

There are generally no market demands for anything radically different - not because things are perfect, but rather because the public at large has no idea what they want, need or could use. The last 2 incarnations of e.g. Microsoft Word added nothing of value to anyone I know (and I went to the trouble of inquiriying with ~50 people in different professions); Microsoft searched hard for features that would justify a new version of Word, and IMHO, just couldn't.

Now is the turn of the OS. XP already adds little to none for 99% of the users over Win2K. The answers about why XP is better I could get revolve around "better hardware support" (semi-valid; but will not be valid in six months time or so when hardware that came out after WinXP becomes ubiqutous), "built in ZIP and MP3 support" (which I don't consider valid, because most people will end up installing e.g. DivX codecs, and other pieces). Basically, if you can manage with XP you'll manage just as well with 2K (minus the teletubbies theme; or, if you really insist, install WindowBlinds). But XP does take some control away from the user - see, e.g.,  [ http://www.hevanet.com/peace/microsoft.htm ].

The next Microsoft OS is going to be based on SQLServer core. I suppose it will find more use than, e.g., the multiple file streams in NTFS, but not much. It is rumoured to require new hardware (Which means more new software, etc.)

If users don't buy new software, the vendor is in trouble (but, if the old software is satisfiying, the users don't earn much). Except a skinnable new language, what about .NET wasn't available already in other frameworks?

No, I can't quantify that numerically -- it has different value for different people (e.g., I value my freedom to choose highly. Do you value yours?). And it may not apply to other people. But it's a simpler explanation occam wise, and it makes perfect market sense.

Ori Berger
Sunday, December 15, 2002

lkjdhsfljka sdjfkl jfakhjahjdkl hk  lkdflrk;a hltioutn j;lj laj;jl; ggl;jg;jgkl ;r';tjiri4e4u 0 

Kendrick Anthony Miranda
Tuesday, October 07, 2003

*  Recent Topics

*  Fog Creek Home