Metadata-Version: 2.4
Name: sumy
Version: 0.12.0
Summary: Module for automatic summarization of text documents and HTML pages.
Project-URL: Homepage, https://github.com/miso-belica/sumy
Project-URL: Repository, https://github.com/miso-belica/sumy
Author-email: Mišo Belica <miso.belica@gmail.com>
License: Apache License, Version 2.0
License-File: LICENSE.md
Keywords: LSA,LexRank,NLP,TextRank,automatic summarization,data mining,data reduction,latent semantic analysis,natural language processing,web-data extraction
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Natural Language :: Arabic
Classifier: Natural Language :: Chinese (Simplified)
Classifier: Natural Language :: Czech
Classifier: Natural Language :: English
Classifier: Natural Language :: French
Classifier: Natural Language :: German
Classifier: Natural Language :: Greek
Classifier: Natural Language :: Hebrew
Classifier: Natural Language :: Italian
Classifier: Natural Language :: Japanese
Classifier: Natural Language :: Polish
Classifier: Natural Language :: Portuguese
Classifier: Natural Language :: Slovak
Classifier: Natural Language :: Spanish
Classifier: Natural Language :: Swedish
Classifier: Natural Language :: Thai
Classifier: Natural Language :: Ukrainian
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
Classifier: Topic :: Education
Classifier: Topic :: Internet
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Text Processing :: Filters
Classifier: Topic :: Text Processing :: Linguistic
Classifier: Topic :: Text Processing :: Markup :: HTML
Requires-Python: >=3.8
Requires-Dist: breadability>=0.1.20
Requires-Dist: docopt-ng>=0.6.1
Requires-Dist: lxml-html-clean
Requires-Dist: nltk>=3.0.2
Requires-Dist: pycountry>=18.2.23
Requires-Dist: requests>=2.7.0
Requires-Dist: setuptools>=65.0.0
Provides-Extra: arabic
Requires-Dist: pyarabic; extra == 'arabic'
Provides-Extra: chinese
Requires-Dist: jieba; extra == 'chinese'
Provides-Extra: greek
Requires-Dist: greek-stemmer-pos; extra == 'greek'
Provides-Extra: hebrew
Requires-Dist: hebrew-tokenizer; extra == 'hebrew'
Provides-Extra: japanese
Requires-Dist: tinysegmenter; extra == 'japanese'
Provides-Extra: korean
Requires-Dist: konlpy; extra == 'korean'
Provides-Extra: lexrank
Requires-Dist: numpy; extra == 'lexrank'
Provides-Extra: lsa
Requires-Dist: numpy; extra == 'lsa'
Provides-Extra: polish
Requires-Dist: pystempel; extra == 'polish'
Provides-Extra: thai
Requires-Dist: pythainlp; extra == 'thai'
Description-Content-Type: text/markdown

# Automatic text summarizer


[![image](https://github.com/miso-belica/sumy/actions/workflows/run-tests.yml/badge.svg)](https://github.com/miso-belica/sumy/actions/workflows/run-tests.yml)
[![GitPod Ready-to-Code](https://img.shields.io/badge/Gitpod-Ready--to--Code-blue?logo=gitpod)](https://gitpod.io/#https://github.com/miso-belica/sumy) 

Simple library and command line utility for extracting summary from HTML
pages or plain texts. The package also contains simple evaluation
framework for text summaries. Implemented summarization methods are described in the [documentation](docs/summarizators.md). I also maintain a list of [alternative implementations](docs/alternatives.md) of the summarizers in various programming languages.

## Is my natural language supported?
There is a [good chance](docs/index.md#Tokenizer) it is. But if not it is [not too hard to add](docs/how-to-add-new-language.md) it.

## Installation

Make sure you have [Python](http://www.python.org/) 3.8+ installed.

### Using uv (recommended)

[uv](https://docs.astral.sh/uv/) is a modern, fast Python package installer and resolver.

```sh
$ uv pip install sumy
$ uv pip install git+https://github.com/miso-belica/sumy.git  # for the fresh version
```

### Using pip

```sh
$ pip install sumy
$ pip install git+git://github.com/miso-belica/sumy.git  # for the fresh version
```

### For development

If you want to contribute or modify the code:

```sh
$ git clone https://github.com/miso-belica/sumy.git
$ cd sumy
$ uv sync               # Install dependencies (dev group included automatically)
$ uv run pytest         # Run tests
$ uv run sumy --help    # Run sumy using uv
```

## Usage

Thanks to some good soul out there, the easiest way to try sumy is in your browser at https://huggingface.co/spaces/issam9/sumy_space

Sumy contains command line utility for quick summarization of documents.

```sh
$ sumy lex-rank --length=10 --url=https://en.wikipedia.org/wiki/Automatic_summarization # what's summarization?
$ sumy lex-rank --language=uk --length=30 --url=https://uk.wikipedia.org/wiki/Україна
$ sumy luhn --language=czech --url=https://www.zdrojak.cz/clanky/automaticke-zabezpeceni/
$ sumy edmundson --language=czech --length=3% --url=https://cs.wikipedia.org/wiki/Bitva_u_Lipan
$ sumy --help # for more info
```

Various evaluation methods for some summarization method can be executed
by commands below:

```sh
$ sumy_eval lex-rank reference_summary.txt --url=https://en.wikipedia.org/wiki/Automatic_summarization
$ sumy_eval lsa reference_summary.txt --language=czech --url=https://www.zdrojak.cz/clanky/automaticke-zabezpeceni/
$ sumy_eval edmundson reference_summary.txt --language=czech --url=https://cs.wikipedia.org/wiki/Bitva_u_Lipan
$ sumy_eval --help # for more info
```

If you don't want to bother by the installation, you can try it as a container.

```sh
$ docker run --rm misobelica/sumy lex-rank --length=10 --url=https://en.wikipedia.org/wiki/Automatic_summarization
```

## Python API

Or you can use sumy like a library in your project. Create file `sumy_example.py` ([don't name it `sumy.py`](https://stackoverflow.com/questions/41334622/python-sumy-no-module-named-sumy-parsers-html)) with the code below to test it.

```python
# -*- coding: utf-8 -*-

from __future__ import absolute_import
from __future__ import division, print_function, unicode_literals

from sumy.parsers.html import HtmlParser
from sumy.parsers.plaintext import PlaintextParser
from sumy.nlp.tokenizers import Tokenizer
from sumy.summarizers.lsa import LsaSummarizer as Summarizer
from sumy.nlp.stemmers import Stemmer
from sumy.utils import get_stop_words


LANGUAGE = "english"
SENTENCES_COUNT = 10


if __name__ == "__main__":
    url = "https://en.wikipedia.org/wiki/Automatic_summarization"
    parser = HtmlParser.from_url(url, Tokenizer(LANGUAGE))
    # or for plain text files
    # parser = PlaintextParser.from_file("document.txt", Tokenizer(LANGUAGE))
    # parser = PlaintextParser.from_string("Check this out.", Tokenizer(LANGUAGE))
    stemmer = Stemmer(LANGUAGE)

    summarizer = Summarizer(stemmer)
    summarizer.stop_words = get_stop_words(LANGUAGE)

    for sentence in summarizer(parser.document, SENTENCES_COUNT):
        print(sentence)
```

## Interesting projects using sumy

I found some interesting projects while browsing the internet or sometimes people wrote me an e-mail with questions, and I was curious how they use the sumy :)

* [Learning to generate questions from text](https://software.intel.com/en-us/articles/using-natural-language-processing-for-smart-question-generation) - https://github.com/adityasarvaiya/Automatic_Question_Generation
* Summarize your video to any duration - https://github.com/aswanthkoleri/VideoMash and similar https://github.com/OpenGenus/vidsum
* Tool for collectively summarizing large discussions - https://github.com/amyxzhang/wikum
* AutoTL;DR bot for [Lemmy](https://en.wikipedia.org/wiki/Lemmy_(software)) uses sumy: https://github.com/RikudouSage/LemmyAutoTldrBot
