who we are

We are a project in Germany sponsored by netluchs.de
and SuMa-eV (an association
supporting the free information access). Because of our limited ressources we concentrate
on our national audience. But international partners are also
welcome, even if our work is only partially translated.

important note: We are no multi-billion-company! Here are
humans, doing their work with much enthusiasm in their spare time. So
please respect the netiquette!

our vision

the whole is more than the sum of its parts

We want to offer a platform to allow users to build their own
customized searchengines. We call them "minisearchers". In contrast
to our competitors, there won't be any precompiled index when you
start (including all the spam). If the user enters a new URL, we
start to crawl the site. Starting up a new index takes slightly longer
- but you really get just the sites
you want.

You can integrate the minisearcher in your website or use it just for
yourself. There are some constraints concerning the load of our
servers which need to be discussed individually. By now we scaled some
special solutions up to several million searches per month.

We aggregate the minisearchers to a global index, which is of very
hight quality. The index is offered to other SuMa-eV-projects like metager2 or special strategic partners
like metager.de.

what we offer

Opencrawl is a platform for several "minisearchers". You get a (simple)
admin-interface where you add your URLs. Each URL gets a "search-scope"
like "subtree", "all the domain" or "domain and all subdomains". Then
our crawler builds step by step an index only with theese pages. You can
can query this index using a http-get-query. The results are returned in
the xml/opensearch format.

the limitations

Please note, that the service is run by a NGO. With building your index
and offering it to you without charges, there are serveral constraints:

we need the permission to use this index for our searchenginges too.

we need to know something about the "load" caused by your
minisearcher. One very critical parameter is the number of
search-queries per month, you are going to expect.

we have to know something about you. We have to obey to the German
laws. We won't index sites covering the themes "sex, porn, drugs,
gambling, nazi, anti democratic, terrorism, copyrighted material"
and any other illegal content.

i want to be a part of it

If you have read so far and still want to be a part of our project,
the next step is pretty simple: just send an email to
info@opencrawl.de and tell us

Who are you?

What do you want to crawl?

How do you want to use the data?

How many search requests per month do you expect?

Because of the high volume of mailspam: use as subject of your mail
"Opencrawl - international"