Arabic Stop words
=================

Description
-----------

It's not easy to detemine the stop words, and in other hand, stop words
differs according to the case, for this purpos, we propose a classified
list which can be parametered by developper.

The Word list contains only wonds in its commun forms, and we have
generated all forms by a script.

It can used as library 'see section arabicstopwords library'

Files
-----

-  data/ : contains data of stopwords
-  data/classified/stopwords.ods: data in LibreOffice format with more
   valuble informations, and classified stopwords
-  docs: docs files
-  scripts: scripts used to generate all forms, and file formats

## Data Structure
-----------------

All forms data .ODS/CSV file - 1st field : unvocalised word ( في) - 2nd
field : unvocalised stemmed word with -'-' between affixes: e.g.
ف-ب-خمسين-ي

.. figure:: doc/images/stopwords.png
   :alt: Stopwords Example

   Stopwords Example

Minimal classified data .ODS/CSV file - 1st field : unvocalised word (
في) - 2nd field : type of the word: e.g. حرف - 3rd field : class of word
: e.g. preposition

| Affixation infomration in other fields: - 4th field : AIN in arabic ,
  if word accept Conjuction 'العطف', '*' else - 5th field : TEH in
  arabic , if word accept definate article 'ال التعريف', '*' else - 6th
  field : JEEM in arabic , if word accept preposition article 'حروف الجر
  المتصلة', '*' else
  - 7th field : DAD in arabic , if word accept IDAFA articles 'الضمائر
  المتصلة', '*' else
| - 7th field : SAD in arabic , if word accept verb conjugation articles
  'التصريف', '*' else
  - 8th field : LAM in arabic , if word accept LAM QASAM articles 'لام
  القسم', '*' else
| - 8th field : MEEM in arabic , if word has ALEF LAM as definition
  article 'معرف', '\*' else

How to customize stop word list
-------------------------------

-  check the minimal form data file (stopwords.csv)
-  comment by "#" all words which you don't need
-  run

   ::

       make

-  catch the output of script in releases folder.

How to update data
------------------

-  check if the word doesn't exist in the minimal form data file (
   classified/stopwords.ods)
-  add affixation information
-  run

   ::

       make

-  catch the output of script in releases folder.

Arabic Stopwords Library
------------------------

install
~~~~~~~

.. code:: shell

    pip install arabicstopwords

usage
~~~~~

-  test if a word is stop

   .. code:: python

       >>> import arabicstopwords.arabicstopwords as stp
       >>> # test if a word is a stop
       ... stp.is_stop(u'ممكن')
       False
       >>> stp.is_stop(u'منكم')
       True

-  stem a stopword \`\`\`python >>> word = u"لعلهم" >>>
   stp.stop\_stem(word) u'لعل'

::

    * list all stop words

            stp.stopwords\_list() ...... len(stp.stopwords\_list())
            13629 len(stp.classed\_stopwords\_list()) 507 \`\`\` \* give
            all forms of a stopword

.. code:: python

    >>> stp.stopword_forms(u"على")
    ....
    >>> len(stp.stopword_forms(u"على"))
    144
