Metadata-Version: 2.1
Name: scrape-up
Version: 1.1.5
Summary: A web-scraping-based python package that enables you to scrape data from various platforms.
Home-page: https://github.com/Clueless-Community/scrape-up
Author: Clueless Community
Author-email: official.cluelesscommunity@gmail.com
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.6
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: bs4
Requires-Dist: requests
Requires-Dist: requests-html
Requires-Dist: beautifulsoup4

# Documentation

## Package Installation

```PowerShell
pip install scrape-up
```

## Examples

### Scrape the numbers of followers on GitHub

```python
# Import the required module
from scrape_up import github

# Instantiate an object with the username provided.
user = github.Users(username="nikhil25803")

# Call the followers function - it will return the number of followers
per user.followers()
```

---

## The platforms we cover.

- [GitHub](https://github.com/Clueless-Community/scrape-up/blob/main/documentation.md#github)
- [GitHub Education](https://github.com/Clueless-Community/scrape-up/blob/main/documentation.md#github-education)
- [Codechef](https://github.com/Clueless-Community/scrape-up/blob/main/documentation.md#codechef)
- [Hacker News](https://github.com/Clueless-Community/scrape-up/blob/main/documentation.md#hacker-news)
- [HackerEarth](https://github.com/Clueless-Community/scrape-up/blob/main/documentation.md#hackerearth)
- [HackerRank](https://github.com/Clueless-Community/scrape-up/blob/main/documentation.md#hackerrank)

### GitHub

```python
from scrape_up import github
```

### Scrape User details

Create an instance of the class `Users`.

```python
user = github.Users(username="nikhil25803")
```

| Methods                       | Details                                                                                            |
| ----------------------------- | -------------------------------------------------------------------------------------------------- |
| `.followers()`                | Returns the number of followers of a user.                                                         |
| `.following()`                | Returns the number of following of a user.                                                         |
| `.get_avatar()`               | Returns the avatar URL of a user.                                                                  |
| `.get_bio()`                  | Returns the bio of a user.                                                                         |
| `.get_repo()`                 | Returns the list of pinned repositories for a user.                                                |
| `.repo_count()`               | Returns the number of Repositories of a user.                                                      |
| `.star_count()`               | Returns the number of stars of a user.                                                             |
| `.get_yearly_contributions()` | Returns the number of contributions made in 365 days frame.                                        |
| `.get_repositories()`         | Returns the list of repositories of a user.                                                        |
| `.get_starred_repos()`        | Return the list of starred repositories of a user.                                                 |
| `.pul_requests()`             | Return the number of pull requests opened in a repository.                                         |
| `.get_followers()`            | Returns the list of followers of a user.                                                           |
| `.get_following_users()`      | Returns the list of users followed by a user.                                                      |
| `.get_achievements()`         | Returns the list of achievements of a user.                                                        |
| `.get_status()`               | Returns the status of a user.                                                                      |
| `.get_contribution_streak()`  | Returns the maximum contribution streak of a user in the past year starting from the current date. |
| `.get_repository_details()`   | Returns the list of repositories with their details.                                               |
| `.get_branch()`               | Returns the list of branches in a repository.                                                      |
| `.get_merged_pull_requests()` | Returns the list of merged pull requests                                                           |
| `.get_open_issues()`          | Returns the list of open issues                                                                    |

### Scrape Repository details

Create an instance of the class `Repository`.

```python
repository = github.Repository(username="nikhil25803", repository_name="scrape-up")
```

| Methods                    | Details                                                                                                                                                        |
| -------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `.fork_count()`            | Returns the number of forks of a repository.                                                                                                                   |
| `.get_contributors()`      | Returns the number of contributors of a repository.                                                                                                            |
| `.topics()`                | Returns the topics of a repository.                                                                                                                            |
| `.pull_requests()`         | Returns the number of pull requests opened in a repository.                                                                                                    |
| `.last_updated_at()`       | Returns the last updated date of a repository.                                                                                                                 |
| `.tags()`                  | Returns the last ten tags of a repository.                                                                                                                     |
| `.releases()`              | Returns the last ten releases of a repository.                                                                                                                 |
| `.issues_count()`          | Returns number of issues in a repository                                                                                                                       |
| `.readme`                  | Saves the readme.md file of the given user to the current working directory. To view the readme.md with a live server, change ".md" to ".html" in "readme.md". |
| `.get_pull_requests_ids()` | Returns all ids of opened pull requests in a repository.                                                                                                       |
| `.get_issues()`            | Returns the list of all open issues in a repository.                                                                                                           |
| `.commits()`               | Returns the number of commits in a repository.                                                                                                                 |
| `.get_readme()`            | Returns & saves README.md file of the special repository (if exists)                                                                                           |
| `.get_environment()`       | Returns the latest deployed link of a repository (if exists).                                                                                                  |
| `.watch_count()`           | Returns the number of watchers of a repository                                                                                                                 |
| `.all_watchers()`          | Returns the username of all watches of a repository                                                                                                            |

### Scrape details of an issue

Create an instance of the class `Issue`

```python
repository = github.Issue(username="nikhil25803", repository_name="scrape-up", issue_number=59)
```

| Methods           | Details                                                                            |
| ----------------- | ---------------------------------------------------------------------------------- |
| `.assignees()`    | Returns the assignees of an issue.                                                 |
| `.labels()`       | Returns the labels of an issue.                                                    |
| `.opened_by()`    | Returns the name of the user, who opened the issue.                                |
| `.title()`        | Returns the title of an issue.                                                     |
| `.is_milestone()` | Returns the milestone, if the issue is part of one or 'No milestone', if it's not. |
| `.opened_at()`    | Returns a string containing the time when the issue was opened in ISO format.      |

### Scrape a pull request details

Create an instance of the class `PullRequest`

```python
pull_request = github.PullRequest(username="nikhil25803", repository_name="scrape-up", pull_request_number=30)
```

| Methods            | Details                                                                    |
| ------------------ | -------------------------------------------------------------------------- |
| `.commits()`       | Returns the number of commits made in a pull request.                      |
| `.title()`         | Returns the title of a pull request.                                       |
| `.labels()`        | Returns all the labels of a pull request, empty list in case of no labels. |
| `.files_changed()` | Returns the number of files changed in a pull request.                     |
| `.reviewers()`     | Return the list of reviewers assigned in a pull request.                   |

### Scrape the details of an organization

Create an instance of class `Organization`

```python
organization = github.Organization(organization_name="Clueless-Community")
```

| Methods                     | Details                                                         |
| --------------------------- | --------------------------------------------------------------- |
| `.top_topics()`             | Returns a list of the most used topics in an organization.      |
| `.followers()`              | Returns the number of followers of an organization.             |
| `.top_languages()`          | Returns the top languages used in an organization.              |
| `.followers()`              | Returns the number of followers of an organization.             |
| `.avatar()`                 | Returns the avatar URL of an organization.                      |
| `.repositories()`           | Returns the list of repositories of an organization.            |
| `.people()`                 | Returns the list of people in an organization.                  |
| `.peoples() `               | Returns the number of people in an organization.                |
| `.get_location() `          | Returns the location of an organization.                        |
| `.repository_details()`     | Returns the list of repositories with their details.            |
| `.pinned_repository()`      | Returns the list of pinned repositories with their details.     |
| `.get_organization_links()` | Returns a dictionary of important website links of a community. |

---

### GitHub Education

```python
from scrape_up import github_education
```

### Scrape user details

Create an instance of the `Events` class.

```py
events = github_education.Events()
```

| Methods         | Details                                                                                                             |
| --------------- | ------------------------------------------------------------------------------------------------------------------- |
| `.get_events()` | Returns the latest events along with their title, image_url, description, date, location, language, tags, and link. |

---

### CodeChef

```python
from scrape_up import codechef
```

### Scrape user details

Create an object of class `Codechef`

```python
user1 = codechef.User(id="username")

```

| Methods         | Details                                                          |
| --------------- | ---------------------------------------------------------------- |
| `get_profile()` | Returns name, username, profile_image_link, rating, details etc. |

---

### Hacker News

```py
from scrape_up import hackernews
```

### Scrape article details

Create an instance of the `Articles` class.

```py
articles = hackernews.Articles()
```

| Methods            | Details                                                                                                    |
| ------------------ | ---------------------------------------------------------------------------------------------------------- |
| `.articles_list()` | Returns the latest articles along with their score, author, author URL, time, comment count, and link.     |
| `.new_articles()`  | Returns the latest new articles along with their score, author, author URL, time, comment count, and link. |
| `.past_articles()` | Returns the past articles along with their score, author, author URL, time, comment count, and link.       |
| `.ask_articles()`  | Returns the asked articles along with their score, author, author URL, time, comment count, and link.      |
| `.show_articles()` | Returns the show articles along with their score, author, author URL, time, comment count, and link.       |
| `.jobs()`          | Returns the jobs along with their time and link.                                                           |

---

### HackerEarth

```py
from scrape_up import hackerearth
```

Create an object of class `Challanges`

```python
hackerearth = hackerearth.Challanges()
```

| Methods          | Details                                                |
| ---------------- | ------------------------------------------------------ |
| `get_upcoming()` | Get the details of upcoming challenges on Hackerearth. |
| `get_ongoing()`  | Get the details of ongoing challenges on Hackerearth.  |
| `get_hiring()`   | Get the details of hiring challenges on Hackerearth.   |

---

### HackerRank

```py
from scrape_up import hackerrank
```

### Scrape user details

Create an object of class `User`.

```python
hackerank = hackerrank.User()
```

| Methods                      | Details                                                                                   |
| ---------------------------- | ----------------------------------------------------------------------------------------- |
| `get_profile(id="username")` | Returns name, username, country, user_type, details, badges, verified_skills, social etc. |
| `get_skills()`               | Returns a list of verified skills and their links                                         |

### Scrape contest details

Create an object of class `Contest`.

```python
hackerank = hackerrank.Contest()
```

| Methods               | Details                                                             |
| --------------------- | ------------------------------------------------------------------- |
| `active_contests()`   | Returns information on active contests like title, status, and link |
| `archived_contests()` | Returns information regarding archived contests                     |

---
