inincompatibility
=================

A dependency-free, code-less ``socket``-based solution for resolving
(**Python** / **conda**) environment incompatibilities.

Usage Guidelines
----------------

**Installation**:

To install the ``inincompatibility`` package, run:

.. code:: shell

   pip install inincompatibility

**Example: Making Your LLMs Callable Like an API**:

First, make your LLMs (e.g.,
`Llama-3.2-1B <https://huggingface.co/meta-llama/Llama-3.2-1B>`__)
callable functions:

.. code:: python

   # your_llm.py
   import torch
   from transformers import pipeline as pl

   m = "meta-llama/Llama-3.2-1B"
   p = pl("text-generation", model=m, device="cuda")

   def llm_qa(msg: list) -> dict:
       return p(msg)[0]["generated_text"]

Next, create another Python file (e.g., ``to_import_ori.py``) to
``import`` the LLM function:

.. code:: python

   # to_import_ori.py
   from your_llm import llm_qa

Then, use the ``inincompatibility`` CLI to generate the necessary
importable code and then run the LLM in its (**Python** / **conda**)
environment:

.. code:: shell

   conda activate llama
   pip install inincompatibility
   python -m inincompatibility -i to_import_ori.py -o to_import.py

A file named ``to_import.py`` (specified by the ``-o`` argument) will be
generated as follows:

.. code:: python

   # Generated by `inincompatibility`.
   # **Not** depend on `incompatibility`. :)
   # You can directly `import` this file in another environment.
   import socket
   import pickle

   BUFFER_SIZE = 65536

   client = socket.socket(2, socket.SOCK_STREAM)
   client.connect(('127.0.0.1', 57849))

   def _func_eval(func, args, kwargs):
       data = pickle.dumps((func, args, kwargs))
       client.sendall(data)
       return pickle.loads(client.recv(BUFFER_SIZE))

   def _inincompatibility_remote_eval(*args, **kwargs):
       return _func_eval('_inincompatibility_remote_eval', args, kwargs)

   def _inincompatibility_remote_exec(*args, **kwargs):
       return _func_eval('_inincompatibility_remote_exec', args, kwargs)

   def llm_qa(*args, **kwargs):
       return _func_eval('llm_qa', args, kwargs)

Now, you can directly ``import`` the generated code in another
(**Python** / **conda**) environment:

.. code:: python

   # main.py
   from to_import import llm_qa
   print(llm_qa('The key to life is'))

Run your main script (e.g., ``main.py``) in the target environment:

.. code:: shell

   conda activate blackbox
   python main.py

For more details, check out the
`sample-llama <https://github.com/userElaina/inincompatibility/tree/main/sample-llama>`__
directory on
`GitHub <https://github.com/userElaina/inincompatibility>`__.

**Example: Additional Samples**:

For more usage examples, visit the
`sample-local <https://github.com/userElaina/inincompatibility/tree/main/sample-local>`__
directory on
`GitHub <https://github.com/userElaina/inincompatibility>`__.
