Imagine a world without Google… it’s not a pretty sight, is it? Worse still, imagine the internet without search engines! There is so much information out there, there would be no way in the world to isolate the answer to even the simplest of questions. That’s why we need web crawlers and that’s also why the Australian National Data Service funded the development of the software, Media Crawler. The ANDS is committed to ensuring that data can be managed and shared effectively so has created an application that can creep into the darkest corners of any file system, extract and map its contents and store them in a database that can be more easily accessed. This functions in the same fundamental way as search engines, with services like Google running millions of crawlers to ensure the information you need is accessible and to hand as soon as you hit that Search button.
Media Crawler is an RCP application, meaning it uses a minimum number of plug-ins to create a Rich Client Platform – a platform designed as the foundation to building almost any client application. It can be used to search through any file and extract the most useful information from it, enabling you to find exactly what you’re looking for in the vast virtual world. The Australian Data Service has funded this application to assist them in the effort to link data to the people who need to access it and improve the efficiency with which they can locate and extract the data they require. Although the data itself might not be new, accessing it means it can be verified and seen with a new perspective that might reveal previously undiscovered information. Media Crawler connects to the internet and uses a Windows Firewall exception which enables it to connect without interference. By indexing innumerable websites worldwide, Media Crawler increases the likelihood of you finding exactly the information you are searching for. In general, most users install this software for a short time period and then uninstall once they have achieved the goal. Consequently, Media Crawler is both easy to install and uninstall.
