As part of the U.S. govt’s anti-terrorism efforts, it initiated Intelligence Community Directive 501, for research into tools and technologies for information-sharing and retrieval for national defence.

This is something that is really innovative in the arena of national security and intelligence processing. I think of the serious advantage that Indian intelligence agencies will derive, in their fight against terrorism and really hope that something like this gets put in place.

I have been looking at Hadoop and it is being used in production for the Yahoo search engine as well as its database needs. This free and open-source technology allows us to buy really cheap, commodity hardware and use it for really complex data mining queries - which can be in any format. Integration of technologies like SAML, will allow fine-grained access control to secure any highly sensitive data. However, the most important advantage will be unifying all the different database/file stores etc. that various security agencies use and therefore help in connecting the dots.

http://www.informationweek.com/news/government/cloud-saas/showArticle.jhtml?articleID=218501405