سال انتشار: ۱۳۸۴
محل انتشار: یازدهمین کنفرانس سالانه انجمن کامپیوتر ایران
تعداد صفحات: ۴
Mohsen Arabsorkhi – Computer Engineering Department, Shiraz University, Shiraz, Iran
Mehrnoush Shamsfard – Computer Engineering Department,Shahid Beheshti University, Tehran, Iran
This paper presents a method for unsupervised Persian morpheme discovery that automatically discover the morphemes of Persian through the analysis of corpora. We utilized a Minimum Description Length (MDL) based algorithm with some improvement and applied it to Persian corpus. Our improvements include utilizing a heuristic function to compute cost more precisely, using threshold to prevent splitting high frequency chunks, exerting penalty for first and last letters, and distinguish between prefixes and suffixes. In comparison with the simple initial algorithm, our approach has raised precision and recall from (48.24, 45.53) to (63.29, 53.19).