We have implemented an improved free text search for the RIPE Database based on Solr. Find the details and a pointer to this new search function in the article below.
Introduction
We currently offer the possibility to do a free text search within the RIPE Database from our website www.ripe.net. All we can say about the current search method is that it "somewhat works". The search functionality is not quite what we are used to when using web searches.
A new site, a new search engine
The new RIPE NCC website, which will be launched early next year, will use a different portal framework for which a new search engine has been implemented. The underlying free text indexing engine for the new website, Solr , is a search framework from the Apache Lucene project which can easily be extended and has a large community backing.
The current free text search on www.ripe.net uses a special set of database text dump files that form the search index. The new Solr indexer uses direct database access to produce the index, which eliminates the extra step of dumping special files.
The new search complies with the RIPE Database Terms and Conditions and therefore doesn't index any personal data. PERSON and ROLE objects are not indexed at all. However when opening the details of the search results, it is easy to navigate to the related PERSON and ROLE object details.
Improvements and new features
Aside from being faster and producing more accurate results, the new search offers some new useful features:
Filter by object type
Any kind of search frequently returns a lot of hits within different object types. We present all results, plus the option to filter results by object type.
Searching for multiple keyword
It is possible to use "Logical Operators" in the search by simply specifying if the search should produce results where results contain
- ALL matching keywords,
- ANY matching keyword or
- only EXACT matches
Search within specific objects and/or attributes
It is possible to restrict the search to only be within certain object types or even within certain attributes within certain objects, using the advanced search. This is a very powerful search feature.
The good news, its available now!
We have published the new free text search on the RIPE Database Lab environment already. This way you can test the service and provide us with some feedback. When the new www.ripe.net site goes live we will integrate this search feature into the new website.
Have a look at:
http://lab.db.ripe.net/portal/free-text/search.htm
As usual, please let us know what you think.
Comments 11
Comments are disabled on articles published more than a year ago. If you'd like to inform us of any issues, please reach out to us via the contact form here.
Anonymous •
<br />This free text search is great - well done!<br /><br />It doesn't seem to extend to poems in the RIPE database, so I find the -i option on the command line very useful still.<br /><br />Regards.<br /><br />Mike Norris
Hide replies
Anonymous •
Dear Mike<br /><br />We will investigate including poems and poetic-form in the next iteration of the service. We excluded them on the first iteration as poems often contain many words, far more than a typical database object. We did not want them to skew the search results. We could add an 'option' to include poems in the search results.<br /><br />Regards<br />Denis Walker<br />Business Analyst<br />RIPE NCC Database Group
K.Rastas •
Hi, the link seems to be broken now, 17.11.2011 ? HTTP 404 error. Regards, Kristian Rastas
Hide replies
Bogdan Dumitrescu •
Dear Kristian, Thank you for your comment. The full-text search has been promoted to a production service and is now available here: http://apps.db.ripe.net/search/full-text.html Best regards, Bogdan Dumitrescu Software Engineer RIPE NCC Database Group
Daniel R •
I'm using your API to do searches, but cannot find a way to search the full text index with it. How do I use the API to search the full text index?
Hide replies
Kaveh Ranjbar •
Our API only supports queries and updates to main RIPE Database Service and full text search is no supported via the API. Kind Regards, Kaveh. --- Kaveh Ranjbar, RIPE Database Group Manager
Niall Donaghy •
Has there been or are there plans to update the API to support this? Kind regards, Niall
Tim Bruijnzeels •
Hi Niall, At this time there are no plans to extend the API to support this. We believe that the primary use case for this is for ad-hoc searches done by humans. But, please let us know if you see a good use case for this functionality in 3rd party tooling or scripts. Kind regards Tim Bruijnzeels Assistent Manager Software Engineering RIPE NCC
Ross •
Hi Is it possible to add an option to export data from the full text search (json xml or just text).... and if it is not possible is there any chance you could make the feature available? Many thanks Ross
Tim Bruijnzeels •
We are currently focussed on improving the usability of the RIPE Database update UI because we see a lot of issues regarding authentication, and the management of contact information, route objects and reverse DNS. Long term we also want to improve the query interface, but for the time being that has a lower priority. Kind regards Tim Bruijnzeels Assistent Manager Software Engineering RIPE NCC
Ricky Beam •
> good use case for this functionality SPAM hunting. Spammer do everything possible to make themselves hard to find. (i.e. they use different handles for anything that's indexed / searchable) The "descr" field is not searchable via any API at RIPE. The full-text search interface is the only way (short of mirroring the entire whois db), but that output is formatted for humans and thus a pain in the ass to process. If 'http://rest.db.ripe.net/search?source=ripe&inverse-attribute=descr&type-filter=route&query-string=...' worked, it wouldn't take HOURS to track a spammer.