I've been working on doing some corpus linguistics for toki pona. In the last hour, I've downloaded 220KB of long (as in paragraph or longer) texts of toki pona. I've substantial samples from Japanese, English, Russian, French, and German speakers.

Much of it is public domain or CC, so I'll repost a zip file shortly. The rest is unknown license, but I'll make it available to anyone who wants it for computational linguistics work, contact me directly and we'll arrange an exchange in a dark alley where copyright doesn't apply. If this was all on a generous license, we could republish it all as a comprehensive toki pona anthology on lulu.

As you might know, toki pona had a burning-of-the-libraries-of-Alexandria moment when Geocities went down. All of the following fan sites are gone.geocities.com/stephentoddpope/tokiponabible.htmlgeocities.com/yohsweb/geocities.com/yves_prudhomme/toki_pona/geocities.com/stephentoddpope/tokiponahomegeocities.com/girlinside123/toki.html

Corpus Research Resources and how you can helpStand and be counted. Please post medium or long texts you have written to the net, preferably with at least one version without interlinear-gloss (ie. translations between each sentence)

Set your work free. If you do post somewhere on the web, please post with a generous license, public domain, CC or GPL documentation so that we can copy your works willy nilly without a guilty conscience. Otherwise we have to buy the linguistic corpus data from our pot dealer in dark alleys.

Post your work to the forum. When you post to this forum, it becomes covered by a CC license per your TOS with jan Sonja.

alasa tawa nimi pi toki pona. I usually can find most toki pona on the net by using toki pona key words, many of them do not exist in any language. There are significant gaps though in finding toki pona from Asian countries, probably because the search engines I know of aren't indexing hangul, chinese, etc.

Thank you. I have fallen way behind on this and never reached your coverage. I do , however, have a lot of shorter items, casual realtime (well, almost) exchanges and the like (I don't have the IRC logs). This does include a few items from the lost sites (I think -- the names are familiar), but not complete sets. Shall I ZIP what I have to you? I haven't the time nor patience at the moment to set up various searches and counts, all of which need doing. as well as all the indexing that will be of use to Sonja as she works on The Book.

janKipo wrote:Thank you. I have fallen way behind on this and never reached your coverage. I do , however, have a lot of shorter items, casual realtime (well, almost) exchanges and the like (I don't have the IRC logs). This does include a few items from the lost sites (I think -- the names are familiar), but not complete sets. Shall I ZIP what I have to you? I haven't the time nor patience at the moment to set up various searches and counts, all of which need doing. as well as all the indexing that will be of use to Sonja as she works on The Book.

Absolutely! With the quantity of good material I've found, I'm optimistic about compiling an anthology in my spare time. Private message me if you need my email address again.

Here is a zip file of all the current redistributable toki pona text I can find. So far 50% of the collected corpus is redistributable.

Shout outs to the following awesome people who have published with a license compatible with republication. Michael Freedman - BY-NC-NDEveryone who posted to wikia - CC-BY-SAJohn Clifford - (by forum post implying he was willing to contribute, but not specific license yet)Joop Kiefte - Public DomainBryant Knight - Public DomainSonja Kisa - BY-NC-SADave Raftery - Creative CommonsRowa Giso (not sure about their name) - AFAIK, these texts are a derivative of B Knight's works, which is now public domain.

Posts to this forum are covered by CC per your TOS with jan Sonja (thanks for pointing that out jan Ote!) I suspect that re-licensing magic only works when the original copyright holder posts, though.

However, as far as I can tell, tokilili, yahoo, live journal, etc are all under copyright of the original contributors, the sites that post TOS usually say that the site gets limited license to run the mailing list or what have you, but there's no redistribution rights for folk like me...but like any copyright, it only matters to the extent that some can afford to enforce. But that is a story for another day.

I excluded anything that was a translation of a copyrighted work and wasn't small enough to be considered fair use.

Once I get my toki pona website up again, I'll publish it there, too.

The next step is to rummage through this and start correcting the grammar and removing extraneous English, although the No-Deriv's licenses on some of the texts worry me about if I can do that.

http://en.tokipona.org/wiki/Copyright wrote:All original text, images, sounds and videos on the Toki Pona website are licensed under the Creative Commons Attribution-Non-Commercial-Share Alike 3.0 Unported Licence. Anything you contribute to this website's wiki and forums will also be published under this licence.

There are some tp texts on my tp site: http://tokipl.wikidot.com/tekstyCurrent license for the site: CC-NC-SA, the same license has been used as for tp Wiki and this forum. Well, janSonja have chosen 'unported' version. Under this license derivative works are allowed, then: corrected and improved versions can be published (from all people mentioned by janMato only M.Freedman used CC-NC-ND).Texts there:

ma tomo Pape -- by Bryant Knight, from Wikipedia article

sike wan -- by Bryant Knight, from his site

jan lawa Oliki -- a modified version of a text by soweli Elepanto, sent to tp forum

http://en.tokipona.org/wiki/Copyright wrote:All original text, images, sounds and videos on the Toki Pona website are licensed under the Creative Commons Attribution-Non-Commercial-Share Alike 3.0 Unported Licence. Anything you contribute to this website's wiki and forums will also be published under this licence.

Thanks for pointing that out! I'll have to take time to scavenge this forum for suitable texts-- it could end up being as important as wikia. The yahoo forums text posted to this forum though, I considering as "unknown license."

Thought it would be worth mentioning here that I've recently brought the Toki Pona language to life over on Glosbe.com. The site originally had only 24 translated words, but I've now beefed the site up with over 1,200 English words and their corresponding word/phrase in Toki Pona. I made sure to provide thorough English definitions for all translated words to ensure there is no semantic ambiguity regarding English words with multiple corresponding Toki Pona words (ex. "cause" could be "kama", "pana", or "tan" based on context). Totally recommend checking out the site as a handy reference and contribution-hub.