A wrapper on hunspell for use in Python
Cython wrapper on Hunspell Dictionary
This repository provides a wrapper on Hunspell to be used natively in Python. The module uses cython to link between the C++ and Python code, with some additional features. There's very little Python overhead as all the heavy lifting is done on the C++ side of the module interface, which gives optimal performance.
The hunspell library will cache any corrections, you can use persistent caching by
adding the use_disk_cache
argument to a Hunspell constructor. Otherwise it uses
in-memory caching.
For the simplest install simply run:
pip install cyhunspell
This will install the hunspell 1.7.0 C++ bindings on your behalf for your platform.
cacheman -- for (optionally asynchronous) persistent caching
The library installs hunspell version 1.7.0. As new version of hunspell become available this library will provide new versions to match.
Spell checking & spell suggestions
Below are some simple examples for how to use the repository.
from hunspell import Hunspell
h = Hunspell()
You now have a usable hunspell object that can make basic queries for you.
h.spell('test') # True
It's a simple task to ask if a particular word is in the dictionary.
h.spell('correct') # True
h.spell('incorect') # False
This will only ever return True or False, and won't give suggestions about why it might be wrong. It also depends on your choice of dictionary.
If you want to get a suggestion from Hunspell, it can provide a corrected label given a basestring input.
h.suggest('incorect') # ('incorrect', 'correction', corrector', 'correct', 'injector')
The suggestions are in sorted order, where the lower the index the closer to the input string.
h.suffix_suggest('do') # ('doing', 'doth', 'doer', 'doings', 'doers', 'doest')
The module can also stem words, providing the stems for pluralization and other inflections.
h.stem('testers') # ('tester', 'test')
h.stem('saves') # ('save',)
Like stemming but return morphological analysis of the input instead.
h.analyze('permanently') # (' st:permanent fl:Y',)
Generate methods are NOT provided at this time due to the 1.7.0 build not producing any results for any inputs, included the documented one. If this is fixed or someone identifies the issue in the call pattern this will be added to the library in the future.
You can also request bulk actions against Hunspell. This will trigger a threaded (without a gil) request to perform the action requested. Currently just 'suggest' and 'stem' are bulk requestable.
h.bulk_suggest(['correct', 'incorect'])
# {'incorect': ('incorrect', 'correction', 'corrector', 'correct', 'injector'), 'correct': ('correct',)}
h.bulk_suffix_suggest(['cat', 'do'])
# {'do': ('doing', 'doth', 'doer', 'doings', 'doers', 'doest'), 'cat': ('cater', 'cats', "cat's", 'caters')}
h.bulk_stem(['stems', 'currencies'])
# {'currencies': ('currency',), 'stems': ('stem',)}
h.bulk_analyze(['dog', 'permanently'])
# {'permanently': (' st:permanent fl:Y',), 'dog': (' st:dog',)}
By default it spawns number of CPUs threads to perform the operation. You can overwrite the concurrency as well.
h.set_concurrency(4) # Four threads will now be used for bulk requests
You can also specify the language or dictionary you wish to use.
h = Hunspell('en_CA') # Canadian English
By default you have the following dictionaries available
However you can download your own and point Hunspell to your custom dictionaries.
h = Hunspell('en_GB-large', hunspell_data_dir='/custom/dicts/dir')
You can also add new dictionaries at runtime by calling the add_dic method.
h.add_dic(os.path.join(PATH_TO, 'special.dic'))
You can add individual words to a dictionary at runtime.
h.add('sillly')
Furthermore you can attach an affix to the word when doing this by providing a second argument
h.add('silllies', "is:plural")
Much like adding, you can remove words.
h.remove(word)
If you want to have Hunspell cache suggestions and stems you can pass it a directory to house such caches.
h = Hunspell(disk_cache_dir='/tmp/hunspell/cache/dir')
This will save all suggestion and stem requests periodically and in the background. The cache will fork after a number of new requests over particular time ranges and save the cache contents while the rest of the program continues onward. Yo'll never have to explicitly save your caches to disk, but you can if you so choose.
h.save_cache()
Otherwise the Hunspell object will cache such requests locally in memory and not persist that memory.
system_encoding='UTF-8'
in the Hunspell
constructor or set the environment variable HUNSPELL_PATH_ENCODING=UTF-8
. Then you must re-encode your hunspell_data_dir
in UTF-8 by passing that argument name to the Hunspell
constructor or setting the HUNSPELL_DATA
environment variable. This is a restriction of Hunspell / Windows operations.Author(s): Tim Rodriguez and Matthew Seal
MIT