Sorry for the delay in responding -- I just saw your question.
The modern, generic name of what you are looking for is a "Digital Asset Management system", a.k.a., a DAM system. It is an huge field of discussion. Many different software products are available ranging from free all the way up to tens of thousands of dollars per year site licenses. In addition, thousands of forum postings have been written on the subject. By far, the quickest way to get an overview of the field is to pick up a copy of this book and study its logic and recommendations thoroughly:
http://www.amazon.com/The-DAM-Book-Management-Photographers/dp/0596523572 .
If you rely on forum threads discussing the various software products, you are likely to run into parochial points of view from people who have some system that "works for them" (eg, say, as a photographer who don't need to file and retrieve AI, INDD and other non-photographic graphics formats), but who don't have a broad base of experience upon which to make recommendations. For example, you will likely see many recommendations for products ranging from Adobe Lightroom, to Google's free image retrieval software, "Picasa". As a photographer, I love it and use both of these, but unless I am mistaken, while they work great for JPGs, TIFFs, PSDs, etc., I pretty sure that neither of them will handle Illustrator and InDesign files.
It sounds like you are going to need an enterprise-level (ie, costly) system that can handle all the different file types that you listed, but you should be very careful in picking one. For example, for many years, I struggled to make a system called "Portfolio" from Extinsis (http://www.extensis.com/ ) only to find that as of a few years ago, although the product was being sold, they weren't doing any continued product development and weren't supporting modern file types (eg, 16 bit, multi-layer TIFFs), so it would lock up whenever it encountered such a file. My understanding is that they have once again become active, but I have no experience with it.
Please let us know if you find anything useful.
Best regards,
Tom M
PS - Here's a thread that lists some of the DAM systems suitable for photographers, as of a few years ago:
http://photo.net/digital-darkroom-forum/00ZKQG
PS#2 - BTW, you absolutely DO NOT want any system that relies on the folder structure that you happen to currently use. This is just too limiting. Instead, you want a system that will find files across multiple drives, across a network, etc. and build a DB based on all the obvious characteristics such as file name, image date, last modification date, keywords (including structured keyword lists), etc.
