"""
The MGT (Mini Generative Transformer) is an algorithm for training language models developed by Sapiens Technology®️ that retrieves textual data and inserts it into a small 'Transformer' model ('mini' model),
so that this 'mini' model can use the retrieved textual data as supplementary informations for the pre-training. The textual information is retrieved based on the current context and is already pre-processed in training previously executed by the MGT architecture itself.
In this way, MGT can considerably expand the knowledge of a conventional 'Transformer' model. Although it performs better in small models, the MGT architecture can also be applied to expand the knowledge of large models.
The MGT architecture algorithm also enables the inference of responses without a pre-trained “Transformer” model, as its architecture provides a native regression function capable of predicting responses with less creativity than conventional “Transformer” models, but with high accuracy.
Training a model on the MGT architecture is orders of magnitude faster and more efficient than training traditional models because it does not use backpropagation, and it can even be trained directly on the CPU.
Despite the advantages, it is important to note that inferences may become relatively slower for training with very large datasets when the "Transformer" model has a long context window at the same time that this context window is completely full.
"""
# --------------------------> A SAPIENS TECHNOLOGY®️ PRODUCTION) <--------------------------
from .mgt import *
"""
The MGT (Mini Generative Transformer) is an algorithm for training language models developed by Sapiens Technology®️ that retrieves textual data and inserts it into a small 'Transformer' model ('mini' model),
so that this 'mini' model can use the retrieved textual data as supplementary informations for the pre-training. The textual information is retrieved based on the current context and is already pre-processed in training previously executed by the MGT architecture itself.
In this way, MGT can considerably expand the knowledge of a conventional 'Transformer' model. Although it performs better in small models, the MGT architecture can also be applied to expand the knowledge of large models.
The MGT architecture algorithm also enables the inference of responses without a pre-trained “Transformer” model, as its architecture provides a native regression function capable of predicting responses with less creativity than conventional “Transformer” models, but with high accuracy.
Training a model on the MGT architecture is orders of magnitude faster and more efficient than training traditional models because it does not use backpropagation, and it can even be trained directly on the CPU.
Despite the advantages, it is important to note that inferences may become relatively slower for training with very large datasets when the "Transformer" model has a long context window at the same time that this context window is completely full.
"""
# --------------------------> A SAPIENS TECHNOLOGY®️ PRODUCTION) <--------------------------
