Sunday, April 30, 2006
United 93, Information Sharing and Metadata
Saw the movie United 93 tonight. Amazing and very intense. I cannot stress enough that it does not disrespect the victims of 9-11. It is also not a political movie. It does not take sides. It's purpose is to first give you the context and then put you in a seat with the heroic passengers of that tragic flight. It succeeds.
Being an information technology professional and the former metadata program manager for the department of homeland security ... I have to admit to you that I left the movie theater very frustrated and pissed off. The movie makes it very plain to see the ramifications of conflicting information, vague information and too much information -- in a crisis the manual systems are too slow!!! What else is it going to take for this to sink in? That brings us squarely into the realm of metadata -- metadata is not confusing or complex. It is contextual information on your data like lineage (do you trust this data?), security (should you see this?), privacy (does this violate a citizen's reasonable expectation for "personal space") efficiency (what is the cycle time from publishing to discovery and use?) and summarization (this is about subject 'x' and it has geographical information so it can be plotted on a map). You cannot have information sharing without metadata. Period.
We solved this problem! The Data Reference Model (DRM) clearly spells out how to assess, organize and create a data architecture that supports information sharing. And yes, it does require a metadata registry/repository. It does require data stores to be treated as "ENTERPRISE ASSETS". Before I left DHS, there was a strategic memo at the very top tier of DHS that stated this and that policy memo was meeting resistance by some components. That is just plain wrong. I sincerely hope that situation has been fixed with the new policy organization that was stood up under the Second Stage Review (2SR).
But there is plenty of blame to go around. Congress did not fund the Metadata Center of Excellence. Period. Why? The unofficial answer was that we needed to leverage the component assets better? Huh? That answer makes no sense because metadata and data standards must be centralized -- that is the definition of a standard -- a single source of truth! How do we fix this? Congress must be better educated as to the purpose and utility of metadata. This means that us technical weanies have to stop bickering amongst ourselves (specifically Clay Shirky's "metacrap" -- I will debunk this article in a series of later posts. Let me clearly state here that his arguments are faulty and specious). There is a technical camp that preaches "google as a silver bullet" as to why we don't need metadata or any form of data tagging. I hope google with their "do no evil" mantra will get off the fence and come down on the right side of this issue. Of course, the implicit metadata that they use in their page-rank algorithm sure as heck is metadata. It is just metadata created by others and not by themselves. Secondly, their ad-sense program definitely uses an ontology. So, anyone that thinks Google is a testament against metadata does not understand Google.
Since, I am doing a bit of bomb-throwing here; let me also state plainly that Microsoft needs to support the Web Ontology Language (OWL) and stop trying to only support things it can control (I will blog more on the details of this later). WinFS without OWL is a poor copy-cat of Apple's spotlight. When will they learn that such a machiavellian attitude is a losing long-term solution. Sure, you can get some points in the short run. You can even become the richest person in the world. But you won't build anything that lasts. You won't inspire anyone. You won't ever be a company that garners respect... those things last.
Let me leave this entry on a bright note -- the FAA has an excellent data management program that includes both a Metadata Registry/Repository and good data policy. Finally, Ben Sliney (who played himself in the movie), is a credit to himself, the FAA and his Country. I especially loved the scene when he grounds all air traffic over the objection of "how much money it will cost". Some people DO GET IT...
Being an information technology professional and the former metadata program manager for the department of homeland security ... I have to admit to you that I left the movie theater very frustrated and pissed off. The movie makes it very plain to see the ramifications of conflicting information, vague information and too much information -- in a crisis the manual systems are too slow!!! What else is it going to take for this to sink in? That brings us squarely into the realm of metadata -- metadata is not confusing or complex. It is contextual information on your data like lineage (do you trust this data?), security (should you see this?), privacy (does this violate a citizen's reasonable expectation for "personal space") efficiency (what is the cycle time from publishing to discovery and use?) and summarization (this is about subject 'x' and it has geographical information so it can be plotted on a map). You cannot have information sharing without metadata. Period.
We solved this problem! The Data Reference Model (DRM) clearly spells out how to assess, organize and create a data architecture that supports information sharing. And yes, it does require a metadata registry/repository. It does require data stores to be treated as "ENTERPRISE ASSETS". Before I left DHS, there was a strategic memo at the very top tier of DHS that stated this and that policy memo was meeting resistance by some components. That is just plain wrong. I sincerely hope that situation has been fixed with the new policy organization that was stood up under the Second Stage Review (2SR).
But there is plenty of blame to go around. Congress did not fund the Metadata Center of Excellence. Period. Why? The unofficial answer was that we needed to leverage the component assets better? Huh? That answer makes no sense because metadata and data standards must be centralized -- that is the definition of a standard -- a single source of truth! How do we fix this? Congress must be better educated as to the purpose and utility of metadata. This means that us technical weanies have to stop bickering amongst ourselves (specifically Clay Shirky's "metacrap" -- I will debunk this article in a series of later posts. Let me clearly state here that his arguments are faulty and specious). There is a technical camp that preaches "google as a silver bullet" as to why we don't need metadata or any form of data tagging. I hope google with their "do no evil" mantra will get off the fence and come down on the right side of this issue. Of course, the implicit metadata that they use in their page-rank algorithm sure as heck is metadata. It is just metadata created by others and not by themselves. Secondly, their ad-sense program definitely uses an ontology. So, anyone that thinks Google is a testament against metadata does not understand Google.
Since, I am doing a bit of bomb-throwing here; let me also state plainly that Microsoft needs to support the Web Ontology Language (OWL) and stop trying to only support things it can control (I will blog more on the details of this later). WinFS without OWL is a poor copy-cat of Apple's spotlight. When will they learn that such a machiavellian attitude is a losing long-term solution. Sure, you can get some points in the short run. You can even become the richest person in the world. But you won't build anything that lasts. You won't inspire anyone. You won't ever be a company that garners respect... those things last.
Let me leave this entry on a bright note -- the FAA has an excellent data management program that includes both a Metadata Registry/Repository and good data policy. Finally, Ben Sliney (who played himself in the movie), is a credit to himself, the FAA and his Country. I especially loved the scene when he grounds all air traffic over the objection of "how much money it will cost". Some people DO GET IT...
