#+OPTIONS: num:nil

TOC: [[#motivation][Motivation]] | [[#demos][*Demos*]] | [[#install][*Install*]] | [[#try-it-out][Try it out]]

Promnesia is a browser extension for [[https://chrome.google.com/webstore/detail/promnesia/kdmegllpofldcpaclldkopnnjjljoiio][Chrome]]/[[https://addons.mozilla.org/en-US/firefox/addon/promnesia][Firefox]]
(including mobile!) which serves as a web surfing copilot, enhancing your browsing history and web exploration experience.

*TLDR*: it lets you explore your browsing history *in context*: where you encountered it, in chat, on Twitter, on Reddit, or just in one of the text files on your computer.
This is unlike most modern browsers, where you can only see when you visited the link.

It allows you to answer different questions about the current web page:

- have I been here before? When? [[https://karlicoss.github.io/promnesia-demos/child-visits.webm][demo (30 s)]] [[https://karlicoss.github.io/promnesia-demos/child-visits-2.webm][demo (40s)]]
- how did I get on it? Which page has led to it? [[https://karlicoss.github.io/promnesia-demos/how_did_i_get_here.webm][demo (40s)]]
- why have I bookmarked it? who sent me this link? Can I just jump to the message? [[https://karlicoss.github.io/promnesia-demos/watch_later.webm][demo (30s)]]
- which links on this page have I already explored? [[https://karlicoss.github.io/promnesia-demos/mark-visited.webm][demo (30s)]],
- which posts from this blog page have I already read? [[https://karlicoss.github.io/promnesia-demos/mark-visited-2.webm][demo (20s)]]
- have I annotated it? [[https://karlicoss.github.io/promnesia-demos/highlights.webm][demo (1m)]]
- how much time have I spent on it? [[https://user-images.githubusercontent.com/291333/82124084-ba040100-9794-11ea-9af9-ee250ebbb473.png][screenshot]]
- and it also works on your phone! [[https://karlicoss.github.io/promnesia-demos/mobile/panel-jbor.png][screenshot]]

You can jump straight to the [[#demos][Demos]] and [[#install][Install]] sections if you're already overexcited.

* Motivation
See a [[https://beepb00p.xyz/promnesia.html][writeup]] on history and the problems Promnesia is solving. Here's a short summary.

- Has it ever occurred to you that you were reading an old bookmark or some lengthy blog post and suddenly realized you had read it already before?
  It would be fairly easy to search in the browser history, however, it is only stored locally for three months.
  # TODO link?
- Or perhaps you even have a habit of annotating and making notes elsewhere? And you wanna know quickly if you have the current page annotated and display the annotations.
  However, most services want you to use their web apps even for viewing highlights. You don't know if you have highlights, unless you waste time checking every page.
- Or you have this feeling that someone sent you this link ages ago, but you don't remember who and where.
- Or you finally got to watch that thing in your 'Watch later' youtube playlist, that's been there for three years, and now you want to know why did you add it in the first place.

Then this tool is for you.

It consists of two parts:

- the extension: talks to the server and neatly displays the history in a sidebar and shows you whether you visited a page
- the server/backend. It can run locally as a simple Python program (you don't need to host it, although you can).

The server regularly indexes (i.e. extracts URLs and other useful information) [[file:doc/SOURCES.org][data sources]] like:

- data exports from online services: Reddit/Twitter/Hackernews/Telegram/Messenger/Hypothesis/Pocket/Instapaper, etc.

  It heavily benefits from [[https://github.com/karlicoss/HPI][HPI]] package to access the data.

- Google Takeout/Activity backups
- Markdown/org-mode/HTML or any other plaintext on your disk
- in general, anything that can be parsed in some way
- you can also add [[file:doc/SOURCES.org::#extending][your own custom sources]], Promnesia is extensible

#+html: <div id="demo"><div>
* Demos
  :PROPERTIES:
  :CUSTOM_ID: demos
  :END:

- [[https://user-images.githubusercontent.com/291333/69828210-3755ac80-121b-11ea-9d1e-e5086cc9feda.png]]

  # TODO might be good to explain what I mean by context and visit..
  - a green eye means that the link was visited before and has some contexts, associated with it
    when you click on it, the sidebar opens with more information
  - You can see that I've sent the link to someone on Telegram (1st sidebar item )
  - I've annotated the link on instapaper and highlights (2nd and 3rd) are shown inline on the page (yellow)
  - I've clipped some content to my personal notes at some point (2nd sidebar item), the selected text was matched and highlighted as well
    If I click the link, it will cause my Emacs to open and jump straight to the clipping within the file.

- [[https://user-images.githubusercontent.com/291333/64424146-2bd16a00-d0a0-11e9-80d2-73cf3b2b60df.PNG]]

  - I chatted with some of my friends and sent them the link at some point (2nd sidebar item)
    Clicking on the link will open Telegram and jump straight to the message where the link was mentioned.
    So you can reply to it without having to search or scroll over the whole chat history.
    # Json is clearly not the most convenient way to go through conversations with friends, but that's a matter of representing chats in a plaintext form. The benefit though is that once you have any sort of grepable source it's super easy to feed it into the plugin.
  - I've tweeted about this link before (4th item)
    Similarly, clicking would jump straight to my tweet.
  - I also have this link annotated via [[https://hypothes.is][Hypothesis]] (1st, 3rd and 5th items)

- More:

  - You can find more short screencasts, demonstrating various features [[https://github.com/karlicoss/promnesia-demos][here]]
  - There are more screenshots [[https://github.com/karlicoss/promnesia/issues/5#issuecomment-619365708][here]]
  - This [[https://beepb00p.xyz/myinfra-roam.html#promnesia][post]] features a video demo of using Promnesia with a Roam Research database
  - Video demos for the mobile extension:
    - [[https://karlicoss.github.io/promnesia-demos/mobile/panel-w3c.webm][showing all references to pages on W3C website in my org-mode notes]]
    - [[https://karlicoss.github.io/promnesia-demos/mobile/mark-visited-reddit.webm][marking already visited links on the page]], to make easier to process Reddit posts

* Install
  :PROPERTIES:
  :CUSTOM_ID: install
  :END:
  
- extension:

  - [[https://chrome.google.com/webstore/detail/promnesia/kdmegllpofldcpaclldkopnnjjljoiio][Chrome]]
  - [[https://addons.mozilla.org/en-US/firefox/addon/promnesia][Firefox]] (including mobile)
  - you can also find 'unpacked' versions in [[https://github.com/karlicoss/promnesia/releases][Releases]]

    It can be useful because Chrome Web Store releases might take days to approve, but in general it's better to prefer the store version.

- backend

  - simplest: install from [[https://pypi.org/project/promnesia][PyPi]]: =pip3 install --user promnesia=
  - alternatively: you can clone this repository and run it as ~scripts/promnesia~

    This is mainly useful for tinkering with the code and writing new modules.

  You might also need some extra dependencies. See [[file:doc/SOURCES.org::#extra-dependencies]["Extra dependencies"]] for more info.
  
* Try it out
The easies is to try out Promnesia is a demo mode, it can give you a sense of what Promnesia is doing with almost no configuration.

# TODO use smth different
1. [[#install][Install]] the extension and the server in case you haven't already
2. Run ~promnesia demo --port 13131 https://github.com/karlicoss/exobrain~

   This clones the repository, ([[https://github.com/karlicoss/exobrain][my personal wiki]] in this case), extracts the URLs, and runs on the port =16789=

   You can also use a path on your local filesystem, or a website URL.

3. After that, visit https://www.gwern.net

   If you press the extension icon, you will see the pages from my blog where I link to articles on Gwern's site.
  
* Setup your config
# TODO mention where they get the database
To get the most benefit from Promnesia, it's best to properly setup your own config, describing the sources you want it to use.
I'm working on improving the documentation on setting the server up, so please feel free to ask if something is unclear!

- create =config.py=

  You can start from the minimal example config (=cp config.py.example config.py=), and then edit it.
  Alternatively, you can borrow bits from a more complete and annotated configuration example here: [[file:doc/config.py]].
 
  The only required settings are:

    - =OUTPUT_DIR=

      This is the directory for the indexer's output. At the moment it's only the =promnesia.sqlite= file.
      You can keep it as =/tmp=, but you might prefer something else.

    - =SOURCES=

      These are the data sources, that will be processed and indexed by Promnesia.

      You can find the list of available sources with more documentation on each of them here: [[file:doc/SOURCES.org][SOURCES]].

      In addition, you can learn more by:

      - reading example config: [[file:doc/config.py]]
      - browsing the code: [[file:src/promnesia/sources/][promnesia/sources]].

- [optional] check the config

  This is useful to quickly check your config: run =promnesia demo --port 13131 --config /path/to/config.py=.

  This will index the data and launch the server immediately, so you can check that everything works as expected in your browser.

- run the indexer: =promnesia index --config /path/to/config.py=

  [[https://github.com/karlicoss/promnesia/issues/20][At the moment]], indexing is *periodic, not realtime*. The best is to run it via cron/systemd once or several times a day.

- run the server: =promnesia serve --db OUTPUT_DIR/promnesia.sqite=.

  You only have to start it once, it will automatically detect index changes.
- [optional] autostart the server with =promnesia install-server=

  This sets it up to start via Systemd, which should be present in most common *Linux* distributions, so it won't work on Mac/Windows at the moment.

  I don't have a Mac nearby, so if you can contribute a Launchd script, or point me to a setup guide, I'll be very grateful!
  I /think/ you can also use cron with =@reboot= attribute, or just create a manual autostart entry.

# TODO Frontend -- mention what settings are possible?
# TODO possibly reuse JS config stub?

* Extension permissions
  
- =file/http/https=: the extension is meant to work on any page, hence such a broad scope.
 
  Migth be optional in the future, and requested on demand if people feel it's worth it
- =tabs=: making the extension work without an explicit user action (the extension is meant to be a passive assistant)

  Might be optional in the future.
- =history=: to use local browsing history

  Might be optional in the future.
- =activeTab=: getting current tab info and adding the sidebar
- =webNavigation=: watching page state changes (to trigger the extension on page load)
- =storage=: for settings
- =contextMenus=: context menu
- =notifications=: showing notifications

# TODO tabs could be optional in the future?  
* More links
- [[file:doc/SOURCES.org][Documentation on the sources]]
- [[file:doc/DEVELOPMENT.org][Developer's guide]]


# TODO document visits/timestamps/contexts?
# TODO how does it work?
