.import:: Any+, boolean+, notnegative+, type+, iterable+
..from: kindnames

.import:: guppy
..from: guppy

.import:: EquivalenceRelation, IdentitySet, IdentitySetSingleton, Kind, UniSet, Use,
KindOfClassFamily, KindOfRetClaSetFamily, KindOfSizeFamily, KindOfTypeFamily,
KindOfInViaFamily, RootStateType, SetOfKind, Stat

..from: heapykinds

.import:: Kind+, ClodoKind+,
loadableiterableofstrings+, loadablefilenamestring+, 
moduleaddress+, modulename+, objectaddress+,
profilefilename+,
relationname+,
SetOfKind+,SetOfClodoKind+,
UniSet+,
typeorclass+, typeorclassexceptdict+, typeorclassoremptytuple+

..from: heapykinds

.and: Use
..constructor: guppy.hpy

..c: From Classifiers

..attr:: Anything
...kind of: Kind
...d: A kind that represents all kinds of objects.

..attr:: Nothing
...kind of: IdentitySet
...d: The empty set.

..attr:: Class
...d: Equivalence relation by class. It defines objects to
to be equivalent when their builtin __class__ attributes are identical.
...kind of: EquivalenceRelation
...mapping
....d: Create a Kind object that represent an equivalence class
of Class.
....returns: KindOfClassFamily
....arg: tc: typeorclass+
.....d: The Python type or class object that the Kind object will represent.

..attr:: Clodo
...d: Equivalence relation by class or dict owner.
...d: It defines objects to be be equivalent
when their classifications, as defined below, are equal.
....ol:
.....li:If the object is an instance of a class, the
	classification will be the class.
.....li:If the object is not a dictionary,
	the classification will be the type of the object.
.....li:The object is a dictionary. The referrers of the
	object are searched to find one that 'owns' the
	dictionary. That is, the dict is a builtin
	__dict__ attribute of the owner. If no such
	owner is found, the object will be the classified
	as a dict with no owner. If the owner is found, it will
	be classified in its turn, by class as defined in
......ref: .mykind.Class
......t:. The object will be classified as a dict
	owned by such a kind of object.


...kind of: EquivalenceRelation
...mapping
....d: Creates an equivalence class of Clodo.
....returns: Kind
....alt
.....arg: tc: typeorclassexceptdict+
......d: If this argument is specified,
it must be or represent a Python type or class object, except that
it may not be the type dict, and the resulting Kind will be the
same as that returned from
.......ref: .mykind.Class
.......t:(tc).
.....key arg: dictof: typeorclassoremptytuple+
......d: If this argument is specified, and is an empty tuple,
the result will be a kind representing a dict that has no owner.
Otherwise the owner kind will be created as by
.......ref: .mykind.Class
.......t:(dictof),
and the result will be a kind representing a dict that is owned
by that class.

......d:
..attr:: Id
...d: This equivalence relation defines objects to be equivalent
only if they are identical, i.e. have the same address.
...kind of: EquivalenceRelation
...mapping
....d: Creates an equivalence class.
....returns: Kind
....arg: address: objectaddress+
.....d: The argument should be the address of an object.
......c:XXX(which must exist in the heap, due to current implementation).

..c
...attr:: Idset

....d: In this equivalence relation, objects are considered equivalent only
if they are identical, i.e. have the same address, just like for
.....ref: .mykind.Id
......t:. However, the equality classes are represented by instances of
IdentitySet instead of some other Kind object. This is to streamline
some operations in the implementation and unify the behaviour of
IdentitySet and Kind objects. In particular,
.....ref: .mykind.Idset
......t: is the only equivalence relation such that

.....p: a.byidset.kind == a
....d: for all IdentitySet objects a.

....kind of: EquivalenceRelation
....mapping

.....d: Create an equivalence class.
.....returns: IdentitySetSingleton
.....arg: node: Any+

..attr:: Module

...d: This equivalence relation defines objects to be equivalent if
they are the same module, or if none of them is a module. Partitioning
a set of objects using this equivalence relation will therefore
result in one singleton set for each module and one set containing all
other objects.

...kind of: EquivalenceRelation
...mapping:
....d: Create a Kind that matches objects to the given module.
....returns: Kind
....draw
.....key arg: name: modulename+
......d: The name may be specified as a string.
.....key arg: at: moduleaddress+
......d: The address may also be specified, as an
alternative or together with the name.

..attr:: Rcs
...d: (Referrer classification set.)
...d: In this equivalence relation, objects are classified by
classifying their referrers.

...d: The classification of the referrers is done using the classifier
of the
....ref: .mykind.Clodo
....t: equivalence relation. The classifications of the
referrers are collected in a set. This set represents the classification
of the object.

...kind of: EquivalenceRelation
...mapping
....d: Create an equivalence class from user-specified classification arguments.
The arguments should together specify a set of Kind objects, each of
which must represent an equivalence class of
.....ref: .mykind.Clodo
.....t:.
....repeat: 0..*
.....alt
......arg: kind: ClodoKind+
.......d: This adds a single Kind to the set of Kinds of referrers.
......arg: sok: SetOfClodoKind+
.......d: This adds each Kind in the sok argument to the total set of Kinds of referrers.
....returns: KindOfRetClaSetFamily
...c
....dwh: Note
It this is unclear I may specify it mathematically
if it is desired.

..attr:: Root
...kind of: RootStateType

...d: This attribute is a symbolic root containing attributes from
which all reachable objects in the heap can be reached. It is the only
value (a singleton) of its kind; see
....ref: heapykinds.RootStateType
....t: for a description of its attributes.

..attr:: Size

...d: In this equivalence relation, objects are classified by memory
size, so each equivalence class represents a particular size of object.

...kind of: EquivalenceRelation
...mapping
....d: Create a Kind representing a particular size.
....arg: size: notnegative+
....returns: KindOfSizeFamily

..attr:: Type
...d: In this equivalence relation, objects are classified by type
so each equivalence class represents objects of a particular type.

...kind of: EquivalenceRelation
...mapping
....d: Create a Kind representing a particular type.
....arg: type: type+
.....d: A Python type object or a representation of it.
....returns: KindOfTypeFamily

..attr:: Unity
...d: In this equivalence relation, all objects are considered equivalent.
There is only one equivalence class.
...kind of: EquivalenceRelation
...mapping
....returns: Kind
.....d: the equivalence class, which is
......ref: .mykind.Anything
......t:.

..attr:: Via
...d: In this equivalence relation, objects are classified by how
they are referred from their referrers, so each equivalence class
represents objects that have a particular set of relations to
their referrers.

...kind of: EquivalenceRelation
...mapping
....d: Create a Kind representing a particular set of referrer
relations.
....repeat: 0..*
.....arg: rel: relationname+
......d: Each argument specifies one referrer relation.
The arguments should be strings and can be of any of the
following forms.
.......p
.......table
........tr
.........th:
..........code: [expression]
.........td: Indexing of a dict, list, tuple (etc). The expression
must be a Python expression that can be evaluated in a local
environment. The environment will contain the builtins and a name
'hp' that is bound to the current
..........ref: .myfile.Use
..........t: instance.
........tr
.........th:
..........code: .attribute
.........td: Getting an attribute from a builtin type or
a slot of a slotted type. (I.E. not an attribute that
is in a dict of an object.)
........tr
.........th
..........code: .f_locals["name"]
.........td: A local variable of a frame.
........tr
.........th
..........code: .f_locals
..........char: nbsp
..........code: ["name"]
.........td: A variable in a CELL of a frame.
Note the space between
..........char: nbsp
..........code: f_locals
..........t: and
..........code: [
..........t:. This is
to distinguish it from ordinary locals, and still
use a syntax that could be used to access those variables directly from Python.
........tr
.........th
..........code: .keys()[integer]
.........td: A key in a dictionary, at the indicated place in its keys().
.........c: XXX __dict__.keys() - not used - should think about this

....returns: KindOfInViaFamily

..attr:: doc
...d

Overview documentation for top level Heapy object.
Provides a listing of the available attributes.
Accessing the attribute name on the doc objects gives further info, eg:

....pre
>>> hp.doc.heap
....t

gives doc for the heap method when hp is the top level Heapy object.

References may be embedded in the documentations. To access a
reference, opening up a web browser with the doc for it one can do eg:

....pre
>>> hp.doc.heap[1]
....t

The reference number 0 is special. If it is provided, it is the
reference to the html doc for the described object itself. So to see
in the web browser the doc for the heap method one can do:

....pre
>>> hp.doc.heap[0]

..method:: findex

...d: Create an equivalence relation based on a sequence of kinds.  The
name is a combination of find and index.  The classification of each
objects is done as follows:

....p: For each kind in the sequence, check whether the object is an
element of that kind. If it is, the classification is the index of
that kind in the sequence. If the end of the sequence is reached, the
classification is the length of the sequence.

...repeat: 0..*
....arg: kind: Kind+
.....d: Each argument specifies the kind in that position in the sequence.
...returns: EquivalenceRelation
....mapping
.....d: Calling the returned equivalence relation creates an equivalence class.
.....arg: index: notnegative+
......d: The position of the matching kind in the sequence of kinds.
The first one has index 0. Specifying the length of the sequence
means that the equivalence class returned is the one where none
of the kinds in the sequence matched.
...dwh: Bugs

Though the Kind objects representing the equivalence classes work with
set operations such as intersection and union, the tests such as
subset and equality do not generally give the expected result.

..method:: heap
...d: Traverse the heap from a root to find all reachable
and visible objects.
...returns: IdentitySet

...d: The objects that belong to an heapy instance are normally made
invisible. This may be modified in various ways.

...dwh: See also
....ref: .mykind.setref
...dwh: See also
....t: The ht argument to
....ref: .myfile.guppy.hpy

..method:: heapg

...dwh: Note
This method is depreciated. The
....ref: .mykind.heapu
....t: method is the recommended replacement.

...d: Finds the objects in the heap that remain after garbage
collection but are
....em: not
....t: reachable from the root except possibly via containers created
by heapy.

...d: This can be used to find objects in extension modules that remain
in memory even though they are gc-collectable and not reachable.

...dwh: See also
....ref: .mykind.setrelheapg

...dwh: Bug
Sometimes it shows objects in the reachable heap as well, if they
are contained in sets created by a previous call to .heapg().
It was tricky to separate out those objects reachable only via
the heapy containers.

...returns: IdentitySet

..method:: heapu

...d: Finds the objects in the heap that remain after garbage
collection but are
....em: not
....t: reachable from the root.

...d: This can be used to find objects in extension modules that remain
in memory even though they are gc-collectable and not reachable.

...returns: Stat
....d: an object containing a statistical summary of the objects found - not the objects themselves. This is to avoid making the objects reachable.

...dwh: See also
....ref: .mykind.setref

..method:: idset
...d: Create a set of objects based on identity.
...arg: nodes: iterable+
....d: The argument must be an iterable and may yield any kind of objects.
...returns: IdentitySet

...dwh: Note
This method is the same as
....ref: .mykind.iso
....t: except for the argument. 

..method:: iso
...d
....strong: AKA
....em: is object
....t: or
....em: identity set of objects
....t:.
...d: Create a set of objects based on identity.
...repeat: 0..*
....arg: node: Any+
.....d: Any kind of objects are valid arguments.
...returns: IdentitySet

...dwh: Example
....pre
>>> from guppy import hpy
>>> hpy().iso(1, 1, 1.0, [], [], {})
Partition of a set of 5 objects. Total size = 220 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0      1  20      136  62       136  62 dict (no owner)
     1      2  40       56  25       192  87 list
     2      1  20       16   7       208  95 float
     3      1  20       12   5       220 100 int
>>> 
....c: end pre

...dwh: Note
This method is the same as
....ref: .mykind.idset
....t: except for the argument.

..method:: load
...d: Load heapy-related data from a serialized form.
Currently it handles data generated by
....ref: .myfile.Stat.dump
....t:.
...alt
....arg: fn: loadablefilenamestring+
.....d: A string argument is treated as a file name.
....arg: fn: loadableiterableofstrings+
.....d: An open file or an iterator will be iterated over enough
to read one package of data, and another call to load will
read the next package.
...draw
....key arg: use_readline : boolean+
.....d: If true, the method will use .readline() instead of
iteration, which may be necessary in case the input comes from a pipe
since otherwise the Python runtime would try to read ahead a big block
before returning the first package of data.
...dwh: Note
....small: Why it is not always using readline is that AFAIK
iteration should normally be used since it is faster as well
as more general.

...returns: .myfile.Stat
....d: typically, one package of statistical data.
....d: Maybe it may return other kinds of data (in the future)
since there is a directive in the file format to choose an
alternative loader.

..method:: monitor
...d: Start an interactive remote monitor.

....p: This can be used to get information about the state, in
particular the memory usage, of separately running Python
processes. It can connect to such processes, and bring up an
interactive console in each process in which Python can be used much
as it is normally used, while the target process under observation
continues to execute as usual, typically under user control in another
window. By observing and comparing the content of the heap of the
target process at different times it is possible to find problems such
as memory leaks, and then investigate the reasons for the alleged
misbehaviours. It is sometimes also possible to test a fix in the
running target process before updating its source code and restarting
it.

....p: The target process needs to be enabled to accept remote
monitoring. This is implemented via a thread that tries to connect to a
particular socket. This thread can be started in the following way:

....c: sigh, .pre makes to much veritcal space
....p
.....code:import guppy.heapy.RM
....p

It is possible to put that command in the
.....code: sitecustomize.py

.....t: file of the installation. This may be practical so that all
Python processes can be monitored, but there may be security issues to
consider since any process with access to the monitoring socket can
take over the processes that accept remote monitoring.

....p: A process may disable remote monitoring of itself in the
following way:

....p
.....c: pre takes too much space
.....code:
from guppy.heapy.Remote import off; off()
.....c: end pre

...dwh: Example
....pre
nicosys [175] solitaire.py &
[1] 25928
nicosys [176] python -c "from guppy import hpy;hpy().monitor()"
<Monitor> 
*** Connection 1 opened ***
<Monitor> h

Documented commands (type help <topic>):
========================================
exit  h  help  int  lc  q  sc

<Monitor> lc
CID PID   ARGV
  1 25928 ['/home/nilsson/bin/solitaire.py']
<Monitor> sc 1
Remote connection 1. To return to Monitor, type <Ctrl-C> or .<RETURN>
<Annex> h

Documented commands (type help <topic>):
========================================
close  h  help  int  isolatest  q  reset  stat

<Annex> stat
Target overview
------------------------------------
target.sys.executable   = /usr/local/bin/python
target.sys.argv         = ['/home/nilsson/bin/solitaire.py']
target.wd               = /var/tmp
target.pid              = 25928
<Annex> int
Interactive console. To return to Annex, type '-'.
>>> hp.heap()
Partition of a set of 21919 objects. Total size = 1558688 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0  11467  52   739904  47    739904  47 str
\...
     9     52   0    27040   2   1436516  92 dict of __main__.Card
<62 more rows. Type e.g. '_.more' to view.>
>>> .
<Monitor> exit
....c: end pre

..method:: pb
...d: Create a Profile Browser window.
...optionals
....arg: filename: profilefilename+
.....d: The name of a file containing profile data. The data should
be in the format as generated by the
......ref: .myfile.Stat
......t:.dump method. For example, you can generate the data file by
doing the following from time to time:
......pre:
>> from guppy import hpy
>> hpy().heap().stat.dump(filename)
......c: end pre

......t: The dump method appends a sample to the end of the file,
containing a summary description of the current heap contents. The
sequence of such samples can be browsed in the Profile Browser window.

...dwh: See also
....ul
.....li
......a: Heapy Profile Browser
.......href=ProfileBrowser.html
.....li
......a: Screenshot
.......href=pbscreen.jpg

..method:: setref
...d: Set a reference point for heap usage measurement.
...d: This applies to both the
....ref: .mykind.heap
....t: and
....ref: .mykind.heapu

....t: methods. The heap() method will only show the objects allocated
after the time setref was called. The heapu() method, since it deals
with summary data and not actual objects, will show the difference of
sizes and counts compared to when setref was called.

..method:: setrelheap
...dwh: Note
This method may become depreciated or reserved for internal use; the
....ref: .mykind.setref
....t: method is the preferred public interface.
...d: Set relative heap view.
...d: With no argument, set a variable in self to the current set of 
objects in the heap. This set will be subtracted from
the returned set in subsequent calls to
....ref: .mykind.heap
....t:. In this way, it will show the heap relative
to its contents when setrelheap was called, so that only
newly allocated objects are included.

...optionals
....arg: reference: UniSet+
.....d: Pass an alternative set here to set the relative heap reference
to that value instead of the current heap. For example, pass
......ref: .mykind.Nothing
......t: to make the heap view show all visible objects again.

..method:: setrelheapg
...dwh: Note
This method is depreciated. Use
....ref: .mykind.setref
....t: or
....ref: .mykind.setrelheapu
....t: instead.

...d: Equivalent to
....ref: .mykind.setrelheap
....t: except it applies to
....ref: .mykind.heapg
....t: instead. I may consider combining these
methods to one.

...optionals
....arg: reference: UniSet+

..method:: setrelheapu
...dwh: Note
This method may become depreciated or reserved for internal use; the
....ref: .mykind.setref
....t: method is the preferred public interface.
...d: Equivalent to
....ref: .mykind.setrelheap
....t: except it applies to
....ref: .mykind.heapu
....t: instead. Both of these methods are combined in the
....ref: .mykind.setref
....t: method.

...optionals
....arg: reference: UniSet+

.kind:: ExtraUse
..subkind of: Use

..d: Extra attributes that maybe not should be part of the official
interface because they may be complicated to describe or are not very
useful or redundant, and they may well be removed later on.

..method:: union
...d: Calculate the union of a sequence of sets.
...arg: sets: iterable+
....d: The iterable must yield objects of kind UniSet.
...returns: UniSet

..method:: sokind
...d: Create a set of kinds

...repeat: 1..*
....arg: kind: Kind+
.....d: Each argument is a kind to add to the set.
.....d: All kinds must have the same family/classifier and operator.
...returns: SetOfKind

..attr:: sonokind
...d: The empty set of kinds.
...kind of: SetOfKind

..method:: tc_adapt
...d:
 Adapt to a type or class.
 Accepts a type or class object, or a string representation
 (at least as) by tc_repr.
...arg: toc: typeorclass+
...c: .returns: 

..method:: tc_repr

..method:: _load_stat

..method:: uniset_from_setcastable

..c:: From View

..attr:: View
..attr:: _hiding_tag_

