A Formal Grammar for Toki Pona

Zach Tomaszewski

ICS661
06 Dec 2012

Toki Pona




[1] - "Toki Pona." http://en.tokipona.org/. Last accessed: 06 Dec 2012
[2] - Knight, Bryant. "o kama sona e toki pona!" http://bknight0.myweb.uga.edu/toki/lesson/lesson0.html. Last accessed: 06 Dec 2012

Project Goal

Language Overview

Previous Work




[3] - Martin, Matthew. "A Machine Parseable Context Free Grammar for Toki Pona." My Suburban Destiny. Blog. http://www.suburbandestiny.com/?p=805. 10 Sept 2010.

My Work

Grammar

S -> Interjection | VocativeS | Sent | YNAnswer

VocativeS -> NP o | NP o , Sent | NP o Pred | o Pred | Conditional o Pred
YNAnswer -> V | V ala

Sent -> SubjPred | Conditional SubjPred | taso SubjPred

Conditional -> SubjPred la | Context la | NP la

SubjPred -> mi Pred | sina Pred | NP_NoMiSina li Pred | CompoundSubj li Pred

CompoundSubj -> NP en CompoundSubj

NP_NoMiSina -> N_NoMiSina | CompNP
NP -> N | CompNP
CompNP -> NMod | NPpi | NP anu NP
NPpi -> NP pi N Modifier | NP pi Name
NMod -> N Modifier | N N | N N Modifier

Modifier -> Mod | Mod Modifier 
Pred -> VP | VP li Pred

VP -> IntransVP | TransVP | VP PrepPh
IntransVP -> Verb | lon NP | tawa NP | Modal lon NP | Modal tawa NP | Modifier | NP
TransVP -> Verb DO | Modal Verb DO
DO -> e NP | e NP DO
Verb -> V | Modal V | V Mod | YnV | Modal YnV | YnV Mod
YnV -> V ala V

PrepPh -> Prep NP | Prep NP PrepPh

Context -> ante | ken 

Modal -> PosModal | PosModal ala | YnModal
PosModal -> kama | ken | wile 
YnModal -> kama ala kama | ken ala ken | wile ala wile

V -> anpa | ante | awen | ijo | ike | jaki | jan | jo | kalama | kama | ken | kepeken
| kule | lape | lawa | lete | lili | lon | lukin | moku | moli | musi | mute | nasa |
olin | open | pakala | pali | pana | pilin | pimeja | pini | poka | pona | seli | sin
| sitelen | sona | suli | suwi | tawa | telo | toki | tomo | tu | unpa | utala | wan
| wawa | weka | wile

N -> mi | sina | N_NoMiSina

N_NoMiSina -> Name | akesi | ala | ale | ali | ante | ijo | ike | ilo | insa | jaki |
jan | jo | kala | kalama | kama | kasi | ken | kili | kiwen | kule | kute | kulupu |
lawa | len | lete | lili | linja | lipu | luka | lupa | ma | mama | mani | meli |
mije | moku | moli | monsi | mun | musi | mute | nanpa | nasin | nena | ni | nimi |
noka | oko | olin | ona | pakala | pali | palisa | pana | pilin | pimeja | pini |
pipi | poki | poka | pona | seli | selo | seme | sewi | sijelo | sike | sinpin |
sitelen | sona | soweli | suli | suno | supa | suwi | tan | tawa | telo | tenpo |
toki | tomo | tu | unpa | uta | utala | walo | wan | waso | wawa | weka | wile

Mod -> ala | ale | ali | ante | awen | ijo | ike | insa | jaki | jan | jelo | kalama
| kama | kin | kiwen | kule | kute | kulupu | laso | lape | lawa | lete | lili | loje
| lukin | mama | meli | mi | mije | moku | moli | monsi | mun | musi | mute | nasa |
ni | olin | ona | pakala | pali | pimeja | pini | poka | pona | sama | seli | seme |
sewi | sike | sin | sina | suli | suwi | taso | tawa | telo | toki | tomo | tu | unpa
| uta | walo | wan | wawa | weka | wike

Prep -> kepeken | lon | poka | sama | tan | tawa 

Interjection -> a | a a | a a a | ala | ike | jaki | mu | o | pakala | pona | toki

Name -> jan 'Name' | ma 'Name' | ma tomo 'Name' | toki 'Name' | soweli 'Name' 

# New words not included here: alasa, esun, pan, kipisi, pu

Produces 3907 CNF rules.

Results

My parser's results vs earlier Martin parser on the same 100 official sentence corpus.

MineMartin
Failed parses: 1 27
Median parses/sentence generated: 2 4
(or 6, without 0 counts)
Sentences with >= 20 parses: 9%
(max: 54, 45)
23%
(max: 382, 183)
Average parses/sentence (dropped max outlier): 6.8 14.5

Lessons Learned

Thank you

Questions?


Example toki pona sentences:

jan li moku e kili.                   # 'the person eats fruit'
mi moku e telo.                       # 'I drink water' 

jan lili li wile kepeken e tomo telo. # 'The child wants to use the restroom'

moli li ike.                          # 'death is bad'
akesi ike li moli e pipi.             # 'the evil reptile/monster killed the bug'
pipi li moli.                         # 'the insect died'
                                      # ('the insect is deadly')? 
                                      # ('the insect is death')??
                           
o lukin e telo moli akesi.            #'look at the venom' ('reptile deadly liquid')
                                      #(not supported by my current grammar)