
######################################################################
#  NNSIGHT .source QUICK PROFILING
######################################################################

>>> Injection component profiling...

============================================================
INJECTION COMPONENT PROFILING
============================================================

Loading model...

Loading weights:   0%|          | 0/148 [00:00<?, ?it/s]
Loading weights:   1%|          | 1/148 [00:00<00:00, 12710.01it/s, Materializing param=transformer.h.0.attn.c_attn.bias]
Loading weights:   1%|          | 1/148 [00:00<00:00, 6754.11it/s, Materializing param=transformer.h.0.attn.c_attn.bias] 
Loading weights:   1%|▏         | 2/148 [00:00<00:00, 6523.02it/s, Materializing param=transformer.h.0.attn.c_attn.weight]
Loading weights:   1%|▏         | 2/148 [00:00<00:00, 5614.86it/s, Materializing param=transformer.h.0.attn.c_attn.weight]
Loading weights:   2%|▏         | 3/148 [00:00<00:00, 6557.02it/s, Materializing param=transformer.h.0.attn.c_proj.bias]  
Loading weights:   2%|▏         | 3/148 [00:00<00:00, 6032.08it/s, Materializing param=transformer.h.0.attn.c_proj.bias]
Loading weights:   3%|▎         | 4/148 [00:00<00:00, 6944.21it/s, Materializing param=transformer.h.0.attn.c_proj.weight]
Loading weights:   3%|▎         | 4/148 [00:00<00:00, 6561.29it/s, Materializing param=transformer.h.0.attn.c_proj.weight]
Loading weights:   3%|▎         | 5/148 [00:00<00:00, 7286.84it/s, Materializing param=transformer.h.0.ln_1.bias]         
Loading weights:   3%|▎         | 5/148 [00:00<00:00, 6932.73it/s, Materializing param=transformer.h.0.ln_1.bias]
Loading weights:   4%|▍         | 6/148 [00:00<00:00, 7532.42it/s, Materializing param=transformer.h.0.ln_1.weight]
Loading weights:   4%|▍         | 6/148 [00:00<00:00, 7161.59it/s, Materializing param=transformer.h.0.ln_1.weight]
Loading weights:   5%|▍         | 7/148 [00:00<00:00, 7718.23it/s, Materializing param=transformer.h.0.ln_2.bias]  
Loading weights:   5%|▍         | 7/148 [00:00<00:00, 7465.07it/s, Materializing param=transformer.h.0.ln_2.bias]
Loading weights:   5%|▌         | 8/148 [00:00<00:00, 6902.78it/s, Materializing param=transformer.h.0.ln_2.weight]
Loading weights:   5%|▌         | 8/148 [00:00<00:00, 6693.48it/s, Materializing param=transformer.h.0.ln_2.weight]
Loading weights:   6%|▌         | 9/148 [00:00<00:00, 1867.92it/s, Materializing param=transformer.h.0.mlp.c_fc.bias]
Loading weights:   6%|▌         | 9/148 [00:00<00:00, 1854.70it/s, Materializing param=transformer.h.0.mlp.c_fc.bias]
Loading weights:   7%|▋         | 10/148 [00:00<00:00, 1995.48it/s, Materializing param=transformer.h.0.mlp.c_fc.weight]
Loading weights:   7%|▋         | 10/148 [00:00<00:00, 1982.19it/s, Materializing param=transformer.h.0.mlp.c_fc.weight]
Loading weights:   7%|▋         | 11/148 [00:00<00:00, 1179.89it/s, Materializing param=transformer.h.0.mlp.c_proj.bias]
Loading weights:   7%|▋         | 11/148 [00:00<00:00, 1175.41it/s, Materializing param=transformer.h.0.mlp.c_proj.bias]
Loading weights:   8%|▊         | 12/148 [00:00<00:00, 1273.25it/s, Materializing param=transformer.h.0.mlp.c_proj.weight]
Loading weights:   8%|▊         | 12/148 [00:00<00:00, 1268.95it/s, Materializing param=transformer.h.0.mlp.c_proj.weight]
Loading weights:   9%|▉         | 13/148 [00:00<00:00, 1365.71it/s, Materializing param=transformer.h.1.attn.c_attn.bias] 
Loading weights:   9%|▉         | 13/148 [00:00<00:00, 1361.07it/s, Materializing param=transformer.h.1.attn.c_attn.bias]
Loading weights:   9%|▉         | 14/148 [00:00<00:00, 1416.35it/s, Materializing param=transformer.h.1.attn.c_attn.weight]
Loading weights:   9%|▉         | 14/148 [00:00<00:00, 1411.61it/s, Materializing param=transformer.h.1.attn.c_attn.weight]
Loading weights:  10%|█         | 15/148 [00:00<00:00, 1502.76it/s, Materializing param=transformer.h.1.attn.c_proj.bias]  
Loading weights:  10%|█         | 15/148 [00:00<00:00, 1498.07it/s, Materializing param=transformer.h.1.attn.c_proj.bias]
Loading weights:  11%|█         | 16/148 [00:00<00:00, 1563.51it/s, Materializing param=transformer.h.1.attn.c_proj.weight]
Loading weights:  11%|█         | 16/148 [00:00<00:00, 1558.10it/s, Materializing param=transformer.h.1.attn.c_proj.weight]
Loading weights:  11%|█▏        | 17/148 [00:00<00:00, 1590.98it/s, Materializing param=transformer.h.1.ln_1.bias]         
Loading weights:  11%|█▏        | 17/148 [00:00<00:00, 1585.61it/s, Materializing param=transformer.h.1.ln_1.bias]
Loading weights:  12%|█▏        | 18/148 [00:00<00:00, 1668.60it/s, Materializing param=transformer.h.1.ln_1.weight]
Loading weights:  12%|█▏        | 18/148 [00:00<00:00, 1663.60it/s, Materializing param=transformer.h.1.ln_1.weight]
Loading weights:  13%|█▎        | 19/148 [00:00<00:00, 1745.98it/s, Materializing param=transformer.h.1.ln_2.bias]  
Loading weights:  13%|█▎        | 19/148 [00:00<00:00, 1741.06it/s, Materializing param=transformer.h.1.ln_2.bias]
Loading weights:  14%|█▎        | 20/148 [00:00<00:00, 1822.42it/s, Materializing param=transformer.h.1.ln_2.weight]
Loading weights:  14%|█▎        | 20/148 [00:00<00:00, 1817.41it/s, Materializing param=transformer.h.1.ln_2.weight]
Loading weights:  14%|█▍        | 21/148 [00:00<00:00, 1637.15it/s, Materializing param=transformer.h.1.mlp.c_fc.bias]
Loading weights:  14%|█▍        | 21/148 [00:00<00:00, 1632.99it/s, Materializing param=transformer.h.1.mlp.c_fc.bias]
Loading weights:  15%|█▍        | 22/148 [00:00<00:00, 1702.49it/s, Materializing param=transformer.h.1.mlp.c_fc.weight]
Loading weights:  15%|█▍        | 22/148 [00:00<00:00, 1698.26it/s, Materializing param=transformer.h.1.mlp.c_fc.weight]
Loading weights:  16%|█▌        | 23/148 [00:00<00:00, 1767.12it/s, Materializing param=transformer.h.1.mlp.c_proj.bias]
Loading weights:  16%|█▌        | 23/148 [00:00<00:00, 1762.83it/s, Materializing param=transformer.h.1.mlp.c_proj.bias]
Loading weights:  16%|█▌        | 24/148 [00:00<00:00, 1831.04it/s, Materializing param=transformer.h.1.mlp.c_proj.weight]
Loading weights:  16%|█▌        | 24/148 [00:00<00:00, 1826.62it/s, Materializing param=transformer.h.1.mlp.c_proj.weight]
Loading weights:  17%|█▋        | 25/148 [00:00<00:00, 1736.14it/s, Materializing param=transformer.h.2.attn.c_attn.bias] 
Loading weights:  17%|█▋        | 25/148 [00:00<00:00, 1732.21it/s, Materializing param=transformer.h.2.attn.c_attn.bias]
Loading weights:  18%|█▊        | 26/148 [00:00<00:00, 1793.71it/s, Materializing param=transformer.h.2.attn.c_attn.weight]
Loading weights:  18%|█▊        | 26/148 [00:00<00:00, 1789.79it/s, Materializing param=transformer.h.2.attn.c_attn.weight]
Loading weights:  18%|█▊        | 27/148 [00:00<00:00, 1750.27it/s, Materializing param=transformer.h.2.attn.c_proj.bias]  
Loading weights:  18%|█▊        | 27/148 [00:00<00:00, 1746.71it/s, Materializing param=transformer.h.2.attn.c_proj.bias]
Loading weights:  19%|█▉        | 28/148 [00:00<00:00, 1804.25it/s, Materializing param=transformer.h.2.attn.c_proj.weight]
Loading weights:  19%|█▉        | 28/148 [00:00<00:00, 1800.85it/s, Materializing param=transformer.h.2.attn.c_proj.weight]
Loading weights:  20%|█▉        | 29/148 [00:00<00:00, 1820.61it/s, Materializing param=transformer.h.2.ln_1.bias]         
Loading weights:  20%|█▉        | 29/148 [00:00<00:00, 1816.58it/s, Materializing param=transformer.h.2.ln_1.bias]
Loading weights:  20%|██        | 30/148 [00:00<00:00, 1848.34it/s, Materializing param=transformer.h.2.ln_1.weight]
Loading weights:  20%|██        | 30/148 [00:00<00:00, 1844.43it/s, Materializing param=transformer.h.2.ln_1.weight]
Loading weights:  21%|██        | 31/148 [00:00<00:00, 1898.79it/s, Materializing param=transformer.h.2.ln_2.bias]  
Loading weights:  21%|██        | 31/148 [00:00<00:00, 1895.19it/s, Materializing param=transformer.h.2.ln_2.bias]
Loading weights:  22%|██▏       | 32/148 [00:00<00:00, 1949.22it/s, Materializing param=transformer.h.2.ln_2.weight]
Loading weights:  22%|██▏       | 32/148 [00:00<00:00, 1945.41it/s, Materializing param=transformer.h.2.ln_2.weight]
Loading weights:  22%|██▏       | 33/148 [00:00<00:00, 1859.43it/s, Materializing param=transformer.h.2.mlp.c_fc.bias]
Loading weights:  22%|██▏       | 33/148 [00:00<00:00, 1856.06it/s, Materializing param=transformer.h.2.mlp.c_fc.bias]
Loading weights:  23%|██▎       | 34/148 [00:00<00:00, 1906.27it/s, Materializing param=transformer.h.2.mlp.c_fc.weight]
Loading weights:  23%|██▎       | 34/148 [00:00<00:00, 1903.09it/s, Materializing param=transformer.h.2.mlp.c_fc.weight]
Loading weights:  24%|██▎       | 35/148 [00:00<00:00, 1953.23it/s, Materializing param=transformer.h.2.mlp.c_proj.bias]
Loading weights:  24%|██▎       | 35/148 [00:00<00:00, 1950.09it/s, Materializing param=transformer.h.2.mlp.c_proj.bias]
Loading weights:  24%|██▍       | 36/148 [00:00<00:00, 2000.01it/s, Materializing param=transformer.h.2.mlp.c_proj.weight]
Loading weights:  24%|██▍       | 36/148 [00:00<00:00, 1996.73it/s, Materializing param=transformer.h.2.mlp.c_proj.weight]
Loading weights:  25%|██▌       | 37/148 [00:00<00:00, 1785.69it/s, Materializing param=transformer.h.3.attn.c_attn.bias] 
Loading weights:  25%|██▌       | 37/148 [00:00<00:00, 1782.78it/s, Materializing param=transformer.h.3.attn.c_attn.bias]
Loading weights:  26%|██▌       | 38/148 [00:00<00:00, 1745.50it/s, Materializing param=transformer.h.3.attn.c_attn.weight]
Loading weights:  26%|██▌       | 38/148 [00:00<00:00, 1742.79it/s, Materializing param=transformer.h.3.attn.c_attn.weight]
Loading weights:  26%|██▋       | 39/148 [00:00<00:00, 1783.68it/s, Materializing param=transformer.h.3.attn.c_proj.bias]  
Loading weights:  26%|██▋       | 39/148 [00:00<00:00, 1781.12it/s, Materializing param=transformer.h.3.attn.c_proj.bias]
Loading weights:  27%|██▋       | 40/148 [00:00<00:00, 1821.75it/s, Materializing param=transformer.h.3.attn.c_proj.weight]
Loading weights:  27%|██▋       | 40/148 [00:00<00:00, 1819.16it/s, Materializing param=transformer.h.3.attn.c_proj.weight]
Loading weights:  28%|██▊       | 41/148 [00:00<00:00, 1794.61it/s, Materializing param=transformer.h.3.ln_1.bias]         
Loading weights:  28%|██▊       | 41/148 [00:00<00:00, 1791.67it/s, Materializing param=transformer.h.3.ln_1.bias]
Loading weights:  28%|██▊       | 42/148 [00:00<00:00, 1830.45it/s, Materializing param=transformer.h.3.ln_1.weight]
Loading weights:  28%|██▊       | 42/148 [00:00<00:00, 1827.75it/s, Materializing param=transformer.h.3.ln_1.weight]
Loading weights:  29%|██▉       | 43/148 [00:00<00:00, 1866.30it/s, Materializing param=transformer.h.3.ln_2.bias]  
Loading weights:  29%|██▉       | 43/148 [00:00<00:00, 1863.81it/s, Materializing param=transformer.h.3.ln_2.bias]
Loading weights:  30%|██▉       | 44/148 [00:00<00:00, 1769.53it/s, Materializing param=transformer.h.3.ln_2.weight]
Loading weights:  30%|██▉       | 44/148 [00:00<00:00, 1767.09it/s, Materializing param=transformer.h.3.ln_2.weight]
Loading weights:  30%|███       | 45/148 [00:00<00:00, 1796.93it/s, Materializing param=transformer.h.3.mlp.c_fc.bias]
Loading weights:  30%|███       | 45/148 [00:00<00:00, 1794.86it/s, Materializing param=transformer.h.3.mlp.c_fc.bias]
Loading weights:  31%|███       | 46/148 [00:00<00:00, 1830.69it/s, Materializing param=transformer.h.3.mlp.c_fc.weight]
Loading weights:  31%|███       | 46/148 [00:00<00:00, 1828.54it/s, Materializing param=transformer.h.3.mlp.c_fc.weight]
Loading weights:  32%|███▏      | 47/148 [00:00<00:00, 1864.17it/s, Materializing param=transformer.h.3.mlp.c_proj.bias]
Loading weights:  32%|███▏      | 47/148 [00:00<00:00, 1862.15it/s, Materializing param=transformer.h.3.mlp.c_proj.bias]
Loading weights:  32%|███▏      | 48/148 [00:00<00:00, 1897.73it/s, Materializing param=transformer.h.3.mlp.c_proj.weight]
Loading weights:  32%|███▏      | 48/148 [00:00<00:00, 1895.57it/s, Materializing param=transformer.h.3.mlp.c_proj.weight]
Loading weights:  33%|███▎      | 49/148 [00:00<00:00, 1859.35it/s, Materializing param=transformer.h.4.attn.c_attn.bias] 
Loading weights:  33%|███▎      | 49/148 [00:00<00:00, 1856.99it/s, Materializing param=transformer.h.4.attn.c_attn.bias]
Loading weights:  34%|███▍      | 50/148 [00:00<00:00, 1880.62it/s, Materializing param=transformer.h.4.attn.c_attn.weight]
Loading weights:  34%|███▍      | 50/148 [00:00<00:00, 1878.31it/s, Materializing param=transformer.h.4.attn.c_attn.weight]
Loading weights:  34%|███▍      | 51/148 [00:00<00:00, 1858.79it/s, Materializing param=transformer.h.4.attn.c_proj.bias]  
Loading weights:  34%|███▍      | 51/148 [00:00<00:00, 1856.77it/s, Materializing param=transformer.h.4.attn.c_proj.bias]
Loading weights:  35%|███▌      | 52/148 [00:00<00:00, 1889.33it/s, Materializing param=transformer.h.4.attn.c_proj.weight]
Loading weights:  35%|███▌      | 52/148 [00:00<00:00, 1887.22it/s, Materializing param=transformer.h.4.attn.c_proj.weight]
Loading weights:  36%|███▌      | 53/148 [00:00<00:00, 1906.99it/s, Materializing param=transformer.h.4.ln_1.bias]         
Loading weights:  36%|███▌      | 53/148 [00:00<00:00, 1904.84it/s, Materializing param=transformer.h.4.ln_1.bias]
Loading weights:  36%|███▋      | 54/148 [00:00<00:00, 1934.66it/s, Materializing param=transformer.h.4.ln_1.weight]
Loading weights:  36%|███▋      | 54/148 [00:00<00:00, 1932.56it/s, Materializing param=transformer.h.4.ln_1.weight]
Loading weights:  37%|███▋      | 55/148 [00:00<00:00, 1957.06it/s, Materializing param=transformer.h.4.ln_2.bias]  
Loading weights:  37%|███▋      | 55/148 [00:00<00:00, 1954.61it/s, Materializing param=transformer.h.4.ln_2.bias]
Loading weights:  38%|███▊      | 56/148 [00:00<00:00, 1985.75it/s, Materializing param=transformer.h.4.ln_2.weight]
Loading weights:  38%|███▊      | 56/148 [00:00<00:00, 1983.51it/s, Materializing param=transformer.h.4.ln_2.weight]
Loading weights:  39%|███▊      | 57/148 [00:00<00:00, 2014.73it/s, Materializing param=transformer.h.4.mlp.c_fc.bias]
Loading weights:  39%|███▊      | 57/148 [00:00<00:00, 2012.60it/s, Materializing param=transformer.h.4.mlp.c_fc.bias]
Loading weights:  39%|███▉      | 58/148 [00:00<00:00, 1973.65it/s, Materializing param=transformer.h.4.mlp.c_fc.weight]
Loading weights:  39%|███▉      | 58/148 [00:00<00:00, 1971.47it/s, Materializing param=transformer.h.4.mlp.c_fc.weight]
Loading weights:  40%|███▉      | 59/148 [00:00<00:00, 2001.28it/s, Materializing param=transformer.h.4.mlp.c_proj.bias]
Loading weights:  40%|███▉      | 59/148 [00:00<00:00, 1999.27it/s, Materializing param=transformer.h.4.mlp.c_proj.bias]
Loading weights:  41%|████      | 60/148 [00:00<00:00, 2021.50it/s, Materializing param=transformer.h.4.mlp.c_proj.weight]
Loading weights:  41%|████      | 60/148 [00:00<00:00, 2019.24it/s, Materializing param=transformer.h.4.mlp.c_proj.weight]
Loading weights:  41%|████      | 61/148 [00:00<00:00, 1982.05it/s, Materializing param=transformer.h.5.attn.c_attn.bias] 
Loading weights:  41%|████      | 61/148 [00:00<00:00, 1979.96it/s, Materializing param=transformer.h.5.attn.c_attn.bias]
Loading weights:  42%|████▏     | 62/148 [00:00<00:00, 2008.30it/s, Materializing param=transformer.h.5.attn.c_attn.weight]
Loading weights:  42%|████▏     | 62/148 [00:00<00:00, 2006.32it/s, Materializing param=transformer.h.5.attn.c_attn.weight]
Loading weights:  43%|████▎     | 63/148 [00:00<00:00, 1975.63it/s, Materializing param=transformer.h.5.attn.c_proj.bias]  
Loading weights:  43%|████▎     | 63/148 [00:00<00:00, 1973.61it/s, Materializing param=transformer.h.5.attn.c_proj.bias]
Loading weights:  43%|████▎     | 64/148 [00:00<00:00, 2000.91it/s, Materializing param=transformer.h.5.attn.c_proj.weight]
Loading weights:  43%|████▎     | 64/148 [00:00<00:00, 1998.97it/s, Materializing param=transformer.h.5.attn.c_proj.weight]
Loading weights:  44%|████▍     | 65/148 [00:00<00:00, 2013.38it/s, Materializing param=transformer.h.5.ln_1.bias]         
Loading weights:  44%|████▍     | 65/148 [00:00<00:00, 2011.37it/s, Materializing param=transformer.h.5.ln_1.bias]
Loading weights:  45%|████▍     | 66/148 [00:00<00:00, 2013.95it/s, Materializing param=transformer.h.5.ln_1.weight]
Loading weights:  45%|████▍     | 66/148 [00:00<00:00, 2011.92it/s, Materializing param=transformer.h.5.ln_1.weight]
Loading weights:  45%|████▌     | 67/148 [00:00<00:00, 2038.71it/s, Materializing param=transformer.h.5.ln_2.bias]  
Loading weights:  45%|████▌     | 67/148 [00:00<00:00, 2036.76it/s, Materializing param=transformer.h.5.ln_2.bias]
Loading weights:  46%|████▌     | 68/148 [00:00<00:00, 2063.39it/s, Materializing param=transformer.h.5.ln_2.weight]
Loading weights:  46%|████▌     | 68/148 [00:00<00:00, 2061.44it/s, Materializing param=transformer.h.5.ln_2.weight]
Loading weights:  47%|████▋     | 69/148 [00:00<00:00, 2088.03it/s, Materializing param=transformer.h.5.mlp.c_fc.bias]
Loading weights:  47%|████▋     | 69/148 [00:00<00:00, 2086.16it/s, Materializing param=transformer.h.5.mlp.c_fc.bias]
Loading weights:  47%|████▋     | 70/148 [00:00<00:00, 2112.59it/s, Materializing param=transformer.h.5.mlp.c_fc.weight]
Loading weights:  47%|████▋     | 70/148 [00:00<00:00, 2110.69it/s, Materializing param=transformer.h.5.mlp.c_fc.weight]
Loading weights:  48%|████▊     | 71/148 [00:00<00:00, 2017.85it/s, Materializing param=transformer.h.5.mlp.c_proj.bias]
Loading weights:  48%|████▊     | 71/148 [00:00<00:00, 2015.97it/s, Materializing param=transformer.h.5.mlp.c_proj.bias]
Loading weights:  49%|████▊     | 72/148 [00:00<00:00, 2040.93it/s, Materializing param=transformer.h.5.mlp.c_proj.weight]
Loading weights:  49%|████▊     | 72/148 [00:00<00:00, 2039.18it/s, Materializing param=transformer.h.5.mlp.c_proj.weight]
Loading weights:  49%|████▉     | 73/148 [00:00<00:00, 1998.73it/s, Materializing param=transformer.h.6.attn.c_attn.bias] 
Loading weights:  49%|████▉     | 73/148 [00:00<00:00, 1996.87it/s, Materializing param=transformer.h.6.attn.c_attn.bias]
Loading weights:  50%|█████     | 74/148 [00:00<00:00, 2014.71it/s, Materializing param=transformer.h.6.attn.c_attn.weight]
Loading weights:  50%|█████     | 74/148 [00:00<00:00, 2012.79it/s, Materializing param=transformer.h.6.attn.c_attn.weight]
Loading weights:  51%|█████     | 75/148 [00:00<00:00, 2036.48it/s, Materializing param=transformer.h.6.attn.c_proj.bias]  
Loading weights:  51%|█████     | 75/148 [00:00<00:00, 2034.74it/s, Materializing param=transformer.h.6.attn.c_proj.bias]
Loading weights:  51%|█████▏    | 76/148 [00:00<00:00, 2058.33it/s, Materializing param=transformer.h.6.attn.c_proj.weight]
Loading weights:  51%|█████▏    | 76/148 [00:00<00:00, 2056.51it/s, Materializing param=transformer.h.6.attn.c_proj.weight]
Loading weights:  52%|█████▏    | 77/148 [00:00<00:00, 2072.92it/s, Materializing param=transformer.h.6.ln_1.bias]         
Loading weights:  52%|█████▏    | 77/148 [00:00<00:00, 2071.09it/s, Materializing param=transformer.h.6.ln_1.bias]
Loading weights:  53%|█████▎    | 78/148 [00:00<00:00, 1917.41it/s, Materializing param=transformer.h.6.ln_1.weight]
Loading weights:  53%|█████▎    | 78/148 [00:00<00:00, 1915.80it/s, Materializing param=transformer.h.6.ln_1.weight]
Loading weights:  53%|█████▎    | 79/148 [00:00<00:00, 1937.46it/s, Materializing param=transformer.h.6.ln_2.bias]  
Loading weights:  53%|█████▎    | 79/148 [00:00<00:00, 1935.94it/s, Materializing param=transformer.h.6.ln_2.bias]
Loading weights:  54%|█████▍    | 80/148 [00:00<00:00, 1957.60it/s, Materializing param=transformer.h.6.ln_2.weight]
Loading weights:  54%|█████▍    | 80/148 [00:00<00:00, 1956.09it/s, Materializing param=transformer.h.6.ln_2.weight]
Loading weights:  55%|█████▍    | 81/148 [00:00<00:00, 1977.65it/s, Materializing param=transformer.h.6.mlp.c_fc.bias]
Loading weights:  55%|█████▍    | 81/148 [00:00<00:00, 1976.07it/s, Materializing param=transformer.h.6.mlp.c_fc.bias]
Loading weights:  55%|█████▌    | 82/148 [00:00<00:00, 1997.43it/s, Materializing param=transformer.h.6.mlp.c_fc.weight]
Loading weights:  55%|█████▌    | 82/148 [00:00<00:00, 1995.55it/s, Materializing param=transformer.h.6.mlp.c_fc.weight]
Loading weights:  56%|█████▌    | 83/148 [00:00<00:00, 2016.80it/s, Materializing param=transformer.h.6.mlp.c_proj.bias]
Loading weights:  56%|█████▌    | 83/148 [00:00<00:00, 2015.34it/s, Materializing param=transformer.h.6.mlp.c_proj.bias]
Loading weights:  57%|█████▋    | 84/148 [00:00<00:00, 2036.66it/s, Materializing param=transformer.h.6.mlp.c_proj.weight]
Loading weights:  57%|█████▋    | 84/148 [00:00<00:00, 2035.11it/s, Materializing param=transformer.h.6.mlp.c_proj.weight]
Loading weights:  57%|█████▋    | 85/148 [00:00<00:00, 2056.34it/s, Materializing param=transformer.h.7.attn.c_attn.bias] 
Loading weights:  57%|█████▋    | 85/148 [00:00<00:00, 2054.79it/s, Materializing param=transformer.h.7.attn.c_attn.bias]
Loading weights:  58%|█████▊    | 86/148 [00:00<00:00, 2075.78it/s, Materializing param=transformer.h.7.attn.c_attn.weight]
Loading weights:  58%|█████▊    | 86/148 [00:00<00:00, 2074.30it/s, Materializing param=transformer.h.7.attn.c_attn.weight]
Loading weights:  59%|█████▉    | 87/148 [00:00<00:00, 2082.22it/s, Materializing param=transformer.h.7.attn.c_proj.bias]  
Loading weights:  59%|█████▉    | 87/148 [00:00<00:00, 2080.70it/s, Materializing param=transformer.h.7.attn.c_proj.bias]
Loading weights:  59%|█████▉    | 88/148 [00:00<00:00, 2101.53it/s, Materializing param=transformer.h.7.attn.c_proj.weight]
Loading weights:  59%|█████▉    | 88/148 [00:00<00:00, 2100.04it/s, Materializing param=transformer.h.7.attn.c_proj.weight]
Loading weights:  60%|██████    | 89/148 [00:00<00:00, 2104.14it/s, Materializing param=transformer.h.7.ln_1.bias]         
Loading weights:  60%|██████    | 89/148 [00:00<00:00, 2102.50it/s, Materializing param=transformer.h.7.ln_1.bias]
Loading weights:  61%|██████    | 90/148 [00:00<00:00, 2123.03it/s, Materializing param=transformer.h.7.ln_1.weight]
Loading weights:  61%|██████    | 90/148 [00:00<00:00, 2121.53it/s, Materializing param=transformer.h.7.ln_1.weight]
Loading weights:  61%|██████▏   | 91/148 [00:00<00:00, 2142.20it/s, Materializing param=transformer.h.7.ln_2.bias]  
Loading weights:  61%|██████▏   | 91/148 [00:00<00:00, 2140.61it/s, Materializing param=transformer.h.7.ln_2.bias]
Loading weights:  62%|██████▏   | 92/148 [00:00<00:00, 2161.23it/s, Materializing param=transformer.h.7.ln_2.weight]
Loading weights:  62%|██████▏   | 92/148 [00:00<00:00, 2159.64it/s, Materializing param=transformer.h.7.ln_2.weight]
Loading weights:  63%|██████▎   | 93/148 [00:00<00:00, 2180.06it/s, Materializing param=transformer.h.7.mlp.c_fc.bias]
Loading weights:  63%|██████▎   | 93/148 [00:00<00:00, 2178.55it/s, Materializing param=transformer.h.7.mlp.c_fc.bias]
Loading weights:  64%|██████▎   | 94/148 [00:00<00:00, 2151.69it/s, Materializing param=transformer.h.7.mlp.c_fc.weight]
Loading weights:  64%|██████▎   | 94/148 [00:00<00:00, 2150.12it/s, Materializing param=transformer.h.7.mlp.c_fc.weight]
Loading weights:  64%|██████▍   | 95/148 [00:00<00:00, 2166.48it/s, Materializing param=transformer.h.7.mlp.c_proj.bias]
Loading weights:  64%|██████▍   | 95/148 [00:00<00:00, 2164.85it/s, Materializing param=transformer.h.7.mlp.c_proj.bias]
Loading weights:  65%|██████▍   | 96/148 [00:00<00:00, 2184.64it/s, Materializing param=transformer.h.7.mlp.c_proj.weight]
Loading weights:  65%|██████▍   | 96/148 [00:00<00:00, 2182.93it/s, Materializing param=transformer.h.7.mlp.c_proj.weight]
Loading weights:  66%|██████▌   | 97/148 [00:00<00:00, 2153.69it/s, Materializing param=transformer.h.8.attn.c_attn.bias] 
Loading weights:  66%|██████▌   | 97/148 [00:00<00:00, 2152.05it/s, Materializing param=transformer.h.8.attn.c_attn.bias]
Loading weights:  66%|██████▌   | 98/148 [00:00<00:00, 2171.15it/s, Materializing param=transformer.h.8.attn.c_attn.weight]
Loading weights:  66%|██████▌   | 98/148 [00:00<00:00, 2169.56it/s, Materializing param=transformer.h.8.attn.c_attn.weight]
Loading weights:  67%|██████▋   | 99/148 [00:00<00:00, 2148.61it/s, Materializing param=transformer.h.8.attn.c_proj.bias]  
Loading weights:  67%|██████▋   | 99/148 [00:00<00:00, 2147.07it/s, Materializing param=transformer.h.8.attn.c_proj.bias]
Loading weights:  68%|██████▊   | 100/148 [00:00<00:00, 2165.74it/s, Materializing param=transformer.h.8.attn.c_proj.weight]
Loading weights:  68%|██████▊   | 100/148 [00:00<00:00, 2164.28it/s, Materializing param=transformer.h.8.attn.c_proj.weight]
Loading weights:  68%|██████▊   | 101/148 [00:00<00:00, 2176.55it/s, Materializing param=transformer.h.8.ln_1.bias]         
Loading weights:  68%|██████▊   | 101/148 [00:00<00:00, 2175.04it/s, Materializing param=transformer.h.8.ln_1.bias]
Loading weights:  69%|██████▉   | 102/148 [00:00<00:00, 2193.72it/s, Materializing param=transformer.h.8.ln_1.weight]
Loading weights:  69%|██████▉   | 102/148 [00:00<00:00, 2192.29it/s, Materializing param=transformer.h.8.ln_1.weight]
Loading weights:  70%|██████▉   | 103/148 [00:00<00:00, 2202.44it/s, Materializing param=transformer.h.8.ln_2.bias]  
Loading weights:  70%|██████▉   | 103/148 [00:00<00:00, 2201.03it/s, Materializing param=transformer.h.8.ln_2.bias]
Loading weights:  70%|███████   | 104/148 [00:00<00:00, 2219.50it/s, Materializing param=transformer.h.8.ln_2.weight]
Loading weights:  70%|███████   | 104/148 [00:00<00:00, 2218.10it/s, Materializing param=transformer.h.8.ln_2.weight]
Loading weights:  71%|███████   | 105/148 [00:00<00:00, 2164.46it/s, Materializing param=transformer.h.8.mlp.c_fc.bias]
Loading weights:  71%|███████   | 105/148 [00:00<00:00, 2163.04it/s, Materializing param=transformer.h.8.mlp.c_fc.bias]
Loading weights:  72%|███████▏  | 106/148 [00:00<00:00, 2180.71it/s, Materializing param=transformer.h.8.mlp.c_fc.weight]
Loading weights:  72%|███████▏  | 106/148 [00:00<00:00, 2179.19it/s, Materializing param=transformer.h.8.mlp.c_fc.weight]
Loading weights:  72%|███████▏  | 107/148 [00:00<00:00, 2196.86it/s, Materializing param=transformer.h.8.mlp.c_proj.bias]
Loading weights:  72%|███████▏  | 107/148 [00:00<00:00, 2195.50it/s, Materializing param=transformer.h.8.mlp.c_proj.bias]
Loading weights:  73%|███████▎  | 108/148 [00:00<00:00, 2169.49it/s, Materializing param=transformer.h.8.mlp.c_proj.weight]
Loading weights:  73%|███████▎  | 108/148 [00:00<00:00, 2168.06it/s, Materializing param=transformer.h.8.mlp.c_proj.weight]
Loading weights:  74%|███████▎  | 109/148 [00:00<00:00, 2182.20it/s, Materializing param=transformer.h.9.attn.c_attn.bias] 
Loading weights:  74%|███████▎  | 109/148 [00:00<00:00, 2180.82it/s, Materializing param=transformer.h.9.attn.c_attn.bias]
Loading weights:  74%|███████▍  | 110/148 [00:00<00:00, 2156.35it/s, Materializing param=transformer.h.9.attn.c_attn.weight]
Loading weights:  74%|███████▍  | 110/148 [00:00<00:00, 2154.40it/s, Materializing param=transformer.h.9.attn.c_attn.weight]
Loading weights:  75%|███████▌  | 111/148 [00:00<00:00, 2170.80it/s, Materializing param=transformer.h.9.attn.c_proj.bias]  
Loading weights:  75%|███████▌  | 111/148 [00:00<00:00, 2169.47it/s, Materializing param=transformer.h.9.attn.c_proj.bias]
Loading weights:  76%|███████▌  | 112/148 [00:00<00:00, 2186.38it/s, Materializing param=transformer.h.9.attn.c_proj.weight]
Loading weights:  76%|███████▌  | 112/148 [00:00<00:00, 2185.09it/s, Materializing param=transformer.h.9.attn.c_proj.weight]
Loading weights:  76%|███████▋  | 113/148 [00:00<00:00, 2189.47it/s, Materializing param=transformer.h.9.ln_1.bias]         
Loading weights:  76%|███████▋  | 113/148 [00:00<00:00, 2188.06it/s, Materializing param=transformer.h.9.ln_1.bias]
Loading weights:  77%|███████▋  | 114/148 [00:00<00:00, 2204.74it/s, Materializing param=transformer.h.9.ln_1.weight]
Loading weights:  77%|███████▋  | 114/148 [00:00<00:00, 2203.43it/s, Materializing param=transformer.h.9.ln_1.weight]
Loading weights:  78%|███████▊  | 115/148 [00:00<00:00, 2220.23it/s, Materializing param=transformer.h.9.ln_2.bias]  
Loading weights:  78%|███████▊  | 115/148 [00:00<00:00, 2218.86it/s, Materializing param=transformer.h.9.ln_2.bias]
Loading weights:  78%|███████▊  | 116/148 [00:00<00:00, 2235.56it/s, Materializing param=transformer.h.9.ln_2.weight]
Loading weights:  78%|███████▊  | 116/148 [00:00<00:00, 2234.24it/s, Materializing param=transformer.h.9.ln_2.weight]
Loading weights:  79%|███████▉  | 117/148 [00:00<00:00, 2250.97it/s, Materializing param=transformer.h.9.mlp.c_fc.bias]
Loading weights:  79%|███████▉  | 117/148 [00:00<00:00, 2249.69it/s, Materializing param=transformer.h.9.mlp.c_fc.bias]
Loading weights:  80%|███████▉  | 118/148 [00:00<00:00, 2260.76it/s, Materializing param=transformer.h.9.mlp.c_fc.weight]
Loading weights:  80%|███████▉  | 118/148 [00:00<00:00, 2259.36it/s, Materializing param=transformer.h.9.mlp.c_fc.weight]
Loading weights:  80%|████████  | 119/148 [00:00<00:00, 2159.51it/s, Materializing param=transformer.h.9.mlp.c_proj.bias]
Loading weights:  80%|████████  | 119/148 [00:00<00:00, 2158.18it/s, Materializing param=transformer.h.9.mlp.c_proj.bias]
Loading weights:  81%|████████  | 120/148 [00:00<00:00, 2174.03it/s, Materializing param=transformer.h.9.mlp.c_proj.weight]
Loading weights:  81%|████████  | 120/148 [00:00<00:00, 2172.86it/s, Materializing param=transformer.h.9.mlp.c_proj.weight]
Loading weights:  82%|████████▏ | 121/148 [00:00<00:00, 2188.73it/s, Materializing param=transformer.h.10.attn.c_attn.bias]
Loading weights:  82%|████████▏ | 121/148 [00:00<00:00, 2187.62it/s, Materializing param=transformer.h.10.attn.c_attn.bias]
Loading weights:  82%|████████▏ | 122/148 [00:00<00:00, 2203.37it/s, Materializing param=transformer.h.10.attn.c_attn.weight]
Loading weights:  82%|████████▏ | 122/148 [00:00<00:00, 2202.15it/s, Materializing param=transformer.h.10.attn.c_attn.weight]
Loading weights:  83%|████████▎ | 123/148 [00:00<00:00, 2190.53it/s, Materializing param=transformer.h.10.attn.c_proj.bias]  
Loading weights:  83%|████████▎ | 123/148 [00:00<00:00, 2189.21it/s, Materializing param=transformer.h.10.attn.c_proj.bias]
Loading weights:  84%|████████▍ | 124/148 [00:00<00:00, 2204.55it/s, Materializing param=transformer.h.10.attn.c_proj.weight]
Loading weights:  84%|████████▍ | 124/148 [00:00<00:00, 2203.24it/s, Materializing param=transformer.h.10.attn.c_proj.weight]
Loading weights:  84%|████████▍ | 125/148 [00:00<00:00, 2209.26it/s, Materializing param=transformer.h.10.ln_1.bias]         
Loading weights:  84%|████████▍ | 125/148 [00:00<00:00, 2208.06it/s, Materializing param=transformer.h.10.ln_1.bias]
Loading weights:  85%|████████▌ | 126/148 [00:00<00:00, 2223.33it/s, Materializing param=transformer.h.10.ln_1.weight]
Loading weights:  85%|████████▌ | 126/148 [00:00<00:00, 2222.15it/s, Materializing param=transformer.h.10.ln_1.weight]
Loading weights:  86%|████████▌ | 127/148 [00:00<00:00, 2183.19it/s, Materializing param=transformer.h.10.ln_2.bias]  
Loading weights:  86%|████████▌ | 127/148 [00:00<00:00, 2181.94it/s, Materializing param=transformer.h.10.ln_2.bias]
Loading weights:  86%|████████▋ | 128/148 [00:00<00:00, 2196.83it/s, Materializing param=transformer.h.10.ln_2.weight]
Loading weights:  86%|████████▋ | 128/148 [00:00<00:00, 2195.72it/s, Materializing param=transformer.h.10.ln_2.weight]
Loading weights:  87%|████████▋ | 129/148 [00:00<00:00, 2210.74it/s, Materializing param=transformer.h.10.mlp.c_fc.bias]
Loading weights:  87%|████████▋ | 129/148 [00:00<00:00, 2209.61it/s, Materializing param=transformer.h.10.mlp.c_fc.bias]
Loading weights:  88%|████████▊ | 130/148 [00:00<00:00, 2224.50it/s, Materializing param=transformer.h.10.mlp.c_fc.weight]
Loading weights:  88%|████████▊ | 130/148 [00:00<00:00, 2223.34it/s, Materializing param=transformer.h.10.mlp.c_fc.weight]
Loading weights:  89%|████████▊ | 131/148 [00:00<00:00, 2235.64it/s, Materializing param=transformer.h.10.mlp.c_proj.bias]
Loading weights:  89%|████████▊ | 131/148 [00:00<00:00, 2234.43it/s, Materializing param=transformer.h.10.mlp.c_proj.bias]
Loading weights:  89%|████████▉ | 132/148 [00:00<00:00, 2208.46it/s, Materializing param=transformer.h.10.mlp.c_proj.weight]
Loading weights:  89%|████████▉ | 132/148 [00:00<00:00, 2207.25it/s, Materializing param=transformer.h.10.mlp.c_proj.weight]
Loading weights:  90%|████████▉ | 133/148 [00:00<00:00, 2221.73it/s, Materializing param=transformer.h.11.attn.c_attn.bias] 
Loading weights:  90%|████████▉ | 133/148 [00:00<00:00, 2220.52it/s, Materializing param=transformer.h.11.attn.c_attn.bias]
Loading weights:  91%|█████████ | 134/148 [00:00<00:00, 2234.88it/s, Materializing param=transformer.h.11.attn.c_attn.weight]
Loading weights:  91%|█████████ | 134/148 [00:00<00:00, 2233.64it/s, Materializing param=transformer.h.11.attn.c_attn.weight]
Loading weights:  91%|█████████ | 135/148 [00:00<00:00, 2225.44it/s, Materializing param=transformer.h.11.attn.c_proj.bias]  
Loading weights:  91%|█████████ | 135/148 [00:00<00:00, 2224.26it/s, Materializing param=transformer.h.11.attn.c_proj.bias]
Loading weights:  92%|█████████▏| 136/148 [00:00<00:00, 2224.74it/s, Materializing param=transformer.h.11.attn.c_proj.weight]
Loading weights:  92%|█████████▏| 136/148 [00:00<00:00, 2223.52it/s, Materializing param=transformer.h.11.attn.c_proj.weight]
Loading weights:  93%|█████████▎| 137/148 [00:00<00:00, 2237.62it/s, Materializing param=transformer.h.11.ln_1.bias]         
Loading weights:  93%|█████████▎| 137/148 [00:00<00:00, 2236.48it/s, Materializing param=transformer.h.11.ln_1.bias]
Loading weights:  93%|█████████▎| 138/148 [00:00<00:00, 2243.86it/s, Materializing param=transformer.h.11.ln_1.weight]
Loading weights:  93%|█████████▎| 138/148 [00:00<00:00, 2242.71it/s, Materializing param=transformer.h.11.ln_1.weight]
Loading weights:  94%|█████████▍| 139/148 [00:00<00:00, 2250.43it/s, Materializing param=transformer.h.11.ln_2.bias]  
Loading weights:  94%|█████████▍| 139/148 [00:00<00:00, 2249.30it/s, Materializing param=transformer.h.11.ln_2.bias]
Loading weights:  95%|█████████▍| 140/148 [00:00<00:00, 2263.36it/s, Materializing param=transformer.h.11.ln_2.weight]
Loading weights:  95%|█████████▍| 140/148 [00:00<00:00, 2262.24it/s, Materializing param=transformer.h.11.ln_2.weight]
Loading weights:  95%|█████████▌| 141/148 [00:00<00:00, 2276.21it/s, Materializing param=transformer.h.11.mlp.c_fc.bias]
Loading weights:  95%|█████████▌| 141/148 [00:00<00:00, 2275.11it/s, Materializing param=transformer.h.11.mlp.c_fc.bias]
Loading weights:  96%|█████████▌| 142/148 [00:00<00:00, 2247.01it/s, Materializing param=transformer.h.11.mlp.c_fc.weight]
Loading weights:  96%|█████████▌| 142/148 [00:00<00:00, 2245.90it/s, Materializing param=transformer.h.11.mlp.c_fc.weight]
Loading weights:  97%|█████████▋| 143/148 [00:00<00:00, 2259.54it/s, Materializing param=transformer.h.11.mlp.c_proj.bias]
Loading weights:  97%|█████████▋| 143/148 [00:00<00:00, 2258.44it/s, Materializing param=transformer.h.11.mlp.c_proj.bias]
Loading weights:  97%|█████████▋| 144/148 [00:00<00:00, 2268.65it/s, Materializing param=transformer.h.11.mlp.c_proj.weight]
Loading weights:  97%|█████████▋| 144/148 [00:00<00:00, 2267.47it/s, Materializing param=transformer.h.11.mlp.c_proj.weight]
Loading weights:  98%|█████████▊| 145/148 [00:00<00:00, 2248.47it/s, Materializing param=transformer.ln_f.bias]             
Loading weights:  98%|█████████▊| 145/148 [00:00<00:00, 2247.35it/s, Materializing param=transformer.ln_f.bias]
Loading weights:  99%|█████████▊| 146/148 [00:00<00:00, 2249.15it/s, Materializing param=transformer.ln_f.weight]
Loading weights:  99%|█████████▊| 146/148 [00:00<00:00, 2248.11it/s, Materializing param=transformer.ln_f.weight]
Loading weights:  99%|█████████▉| 147/148 [00:00<00:00, 2261.70it/s, Materializing param=transformer.wpe.weight] 
Loading weights:  99%|█████████▉| 147/148 [00:00<00:00, 2260.70it/s, Materializing param=transformer.wpe.weight]
Loading weights: 100%|██████████| 148/148 [00:00<00:00, 2274.15it/s, Materializing param=transformer.wte.weight]
Loading weights: 100%|██████████| 148/148 [00:00<00:00, 2273.14it/s, Materializing param=transformer.wte.weight]
Loading weights: 100%|██████████| 148/148 [00:00<00:00, 1678.48it/s, Materializing param=transformer.wte.weight]
GPT2LMHeadModel LOAD REPORT from: gpt2
Key                  | Status     |  | 
---------------------+------------+--+-
h.{0...11}.attn.bias | UNEXPECTED |  | 

Notes:
- UNEXPECTED	:can be ignored when loading from different task/architecture; not ok if you expect identical arch.

[1/4] inspect.getsource()...
  0.509ms

[2/4] ast.parse()...
  0.407ms

[3/4] AST transformation...
  1.603ms
  Operations found: 26

[4/4] Full inject (convert)...
  2.822ms

----------------------------------------
Component breakdown:
  getsource:    0.509ms (20%)
  ast.parse:    0.407ms (16%)
  transform:    1.603ms (64%)
  Sum:          2.519ms
  Full inject:  2.822ms
  Overhead:     0.303ms (compile + exec)

>>> Runtime overhead profiling...

============================================================
RUNTIME OVERHEAD PROFILING
============================================================

Loading model...

Loading weights:   0%|          | 0/148 [00:00<?, ?it/s]
Loading weights:   1%|          | 1/148 [00:00<00:00, 51781.53it/s, Materializing param=transformer.h.0.attn.c_attn.bias]
Loading weights:   1%|          | 1/148 [00:00<00:00, 13751.82it/s, Materializing param=transformer.h.0.attn.c_attn.bias]
Loading weights:   1%|▏         | 2/148 [00:00<00:00, 10394.81it/s, Materializing param=transformer.h.0.attn.c_attn.weight]
Loading weights:   1%|▏         | 2/148 [00:00<00:00, 8533.68it/s, Materializing param=transformer.h.0.attn.c_attn.weight] 
Loading weights:   2%|▏         | 3/148 [00:00<00:00, 4321.05it/s, Materializing param=transformer.h.0.attn.c_proj.bias]  
Loading weights:   2%|▏         | 3/148 [00:00<00:00, 4088.02it/s, Materializing param=transformer.h.0.attn.c_proj.bias]
Loading weights:   3%|▎         | 4/148 [00:00<00:00, 4953.41it/s, Materializing param=transformer.h.0.attn.c_proj.weight]
Loading weights:   3%|▎         | 4/148 [00:00<00:00, 4743.35it/s, Materializing param=transformer.h.0.attn.c_proj.weight]
Loading weights:   3%|▎         | 5/148 [00:00<00:00, 4453.50it/s, Materializing param=transformer.h.0.ln_1.bias]         
Loading weights:   3%|▎         | 5/148 [00:00<00:00, 4315.13it/s, Materializing param=transformer.h.0.ln_1.bias]
Loading weights:   4%|▍         | 6/148 [00:00<00:00, 4885.62it/s, Materializing param=transformer.h.0.ln_1.weight]
Loading weights:   4%|▍         | 6/148 [00:00<00:00, 4761.75it/s, Materializing param=transformer.h.0.ln_1.weight]
Loading weights:   5%|▍         | 7/148 [00:00<00:00, 4447.16it/s, Materializing param=transformer.h.0.ln_2.bias]  
Loading weights:   5%|▍         | 7/148 [00:00<00:00, 4341.93it/s, Materializing param=transformer.h.0.ln_2.bias]
Loading weights:   5%|▌         | 8/148 [00:00<00:00, 1633.53it/s, Materializing param=transformer.h.0.ln_2.weight]
Loading weights:   5%|▌         | 8/148 [00:00<00:00, 1622.08it/s, Materializing param=transformer.h.0.ln_2.weight]
Loading weights:   6%|▌         | 9/148 [00:00<00:00, 1803.83it/s, Materializing param=transformer.h.0.mlp.c_fc.bias]
Loading weights:   6%|▌         | 9/148 [00:00<00:00, 1793.12it/s, Materializing param=transformer.h.0.mlp.c_fc.bias]
Loading weights:   7%|▋         | 10/148 [00:00<00:00, 1969.80it/s, Materializing param=transformer.h.0.mlp.c_fc.weight]
Loading weights:   7%|▋         | 10/148 [00:00<00:00, 1958.31it/s, Materializing param=transformer.h.0.mlp.c_fc.weight]
Loading weights:   7%|▋         | 11/148 [00:00<00:00, 2131.06it/s, Materializing param=transformer.h.0.mlp.c_proj.bias]
Loading weights:   7%|▋         | 11/148 [00:00<00:00, 2119.50it/s, Materializing param=transformer.h.0.mlp.c_proj.bias]
Loading weights:   8%|▊         | 12/148 [00:00<00:00, 2287.18it/s, Materializing param=transformer.h.0.mlp.c_proj.weight]
Loading weights:   8%|▊         | 12/148 [00:00<00:00, 2274.16it/s, Materializing param=transformer.h.0.mlp.c_proj.weight]
Loading weights:   9%|▉         | 13/148 [00:00<00:00, 2438.00it/s, Materializing param=transformer.h.1.attn.c_attn.bias] 
Loading weights:   9%|▉         | 13/148 [00:00<00:00, 2424.88it/s, Materializing param=transformer.h.1.attn.c_attn.bias]
Loading weights:   9%|▉         | 14/148 [00:00<00:00, 2584.97it/s, Materializing param=transformer.h.1.attn.c_attn.weight]
Loading weights:   9%|▉         | 14/148 [00:00<00:00, 2571.28it/s, Materializing param=transformer.h.1.attn.c_attn.weight]
Loading weights:  10%|█         | 15/148 [00:00<00:00, 2638.92it/s, Materializing param=transformer.h.1.attn.c_proj.bias]  
Loading weights:  10%|█         | 15/148 [00:00<00:00, 2623.52it/s, Materializing param=transformer.h.1.attn.c_proj.bias]
Loading weights:  11%|█         | 16/148 [00:00<00:00, 2768.52it/s, Materializing param=transformer.h.1.attn.c_proj.weight]
Loading weights:  11%|█         | 16/148 [00:00<00:00, 2754.43it/s, Materializing param=transformer.h.1.attn.c_proj.weight]
Loading weights:  11%|█▏        | 17/148 [00:00<00:00, 2748.77it/s, Materializing param=transformer.h.1.ln_1.bias]         
Loading weights:  11%|█▏        | 17/148 [00:00<00:00, 2734.12it/s, Materializing param=transformer.h.1.ln_1.bias]
Loading weights:  12%|█▏        | 18/148 [00:00<00:00, 2865.40it/s, Materializing param=transformer.h.1.ln_1.weight]
Loading weights:  12%|█▏        | 18/148 [00:00<00:00, 2851.33it/s, Materializing param=transformer.h.1.ln_1.weight]
Loading weights:  13%|█▎        | 19/148 [00:00<00:00, 2981.47it/s, Materializing param=transformer.h.1.ln_2.bias]  
Loading weights:  13%|█▎        | 19/148 [00:00<00:00, 2966.71it/s, Materializing param=transformer.h.1.ln_2.bias]
Loading weights:  14%|█▎        | 20/148 [00:00<00:00, 2999.79it/s, Materializing param=transformer.h.1.ln_2.weight]
Loading weights:  14%|█▎        | 20/148 [00:00<00:00, 2985.80it/s, Materializing param=transformer.h.1.ln_2.weight]
Loading weights:  14%|█▍        | 21/148 [00:00<00:00, 3107.33it/s, Materializing param=transformer.h.1.mlp.c_fc.bias]
Loading weights:  14%|█▍        | 21/148 [00:00<00:00, 3093.69it/s, Materializing param=transformer.h.1.mlp.c_fc.bias]
Loading weights:  15%|█▍        | 22/148 [00:00<00:00, 3212.91it/s, Materializing param=transformer.h.1.mlp.c_fc.weight]
Loading weights:  15%|█▍        | 22/148 [00:00<00:00, 3199.21it/s, Materializing param=transformer.h.1.mlp.c_fc.weight]
Loading weights:  16%|█▌        | 23/148 [00:00<00:00, 2414.38it/s, Materializing param=transformer.h.1.mlp.c_proj.bias]
Loading weights:  16%|█▌        | 23/148 [00:00<00:00, 2406.13it/s, Materializing param=transformer.h.1.mlp.c_proj.bias]
Loading weights:  16%|█▌        | 24/148 [00:00<00:00, 2494.51it/s, Materializing param=transformer.h.1.mlp.c_proj.weight]
Loading weights:  16%|█▌        | 24/148 [00:00<00:00, 2486.86it/s, Materializing param=transformer.h.1.mlp.c_proj.weight]
Loading weights:  17%|█▋        | 25/148 [00:00<00:00, 2574.14it/s, Materializing param=transformer.h.2.attn.c_attn.bias] 
Loading weights:  17%|█▋        | 25/148 [00:00<00:00, 2566.58it/s, Materializing param=transformer.h.2.attn.c_attn.bias]
Loading weights:  18%|█▊        | 26/148 [00:00<00:00, 2637.86it/s, Materializing param=transformer.h.2.attn.c_attn.weight]
Loading weights:  18%|█▊        | 26/148 [00:00<00:00, 2628.90it/s, Materializing param=transformer.h.2.attn.c_attn.weight]
Loading weights:  18%|█▊        | 27/148 [00:00<00:00, 2561.49it/s, Materializing param=transformer.h.2.attn.c_proj.bias]  
Loading weights:  18%|█▊        | 27/148 [00:00<00:00, 2553.75it/s, Materializing param=transformer.h.2.attn.c_proj.bias]
Loading weights:  19%|█▉        | 28/148 [00:00<00:00, 2634.14it/s, Materializing param=transformer.h.2.attn.c_proj.weight]
Loading weights:  19%|█▉        | 28/148 [00:00<00:00, 2626.54it/s, Materializing param=transformer.h.2.attn.c_proj.weight]
Loading weights:  20%|█▉        | 29/148 [00:00<00:00, 2667.20it/s, Materializing param=transformer.h.2.ln_1.bias]         
Loading weights:  20%|█▉        | 29/148 [00:00<00:00, 2659.21it/s, Materializing param=transformer.h.2.ln_1.bias]
Loading weights:  20%|██        | 30/148 [00:00<00:00, 2673.57it/s, Materializing param=transformer.h.2.ln_1.weight]
Loading weights:  20%|██        | 30/148 [00:00<00:00, 2643.91it/s, Materializing param=transformer.h.2.ln_1.weight]
Loading weights:  21%|██        | 31/148 [00:00<00:00, 2681.78it/s, Materializing param=transformer.h.2.ln_2.bias]  
Loading weights:  21%|██        | 31/148 [00:00<00:00, 2671.15it/s, Materializing param=transformer.h.2.ln_2.bias]
Loading weights:  22%|██▏       | 32/148 [00:00<00:00, 2737.24it/s, Materializing param=transformer.h.2.ln_2.weight]
Loading weights:  22%|██▏       | 32/148 [00:00<00:00, 2728.50it/s, Materializing param=transformer.h.2.ln_2.weight]
Loading weights:  22%|██▏       | 33/148 [00:00<00:00, 2463.77it/s, Materializing param=transformer.h.2.mlp.c_fc.bias]
Loading weights:  22%|██▏       | 33/148 [00:00<00:00, 2452.98it/s, Materializing param=transformer.h.2.mlp.c_fc.bias]
Loading weights:  23%|██▎       | 34/148 [00:00<00:00, 2501.87it/s, Materializing param=transformer.h.2.mlp.c_fc.weight]
Loading weights:  23%|██▎       | 34/148 [00:00<00:00, 2495.30it/s, Materializing param=transformer.h.2.mlp.c_fc.weight]
Loading weights:  24%|██▎       | 35/148 [00:00<00:00, 2555.99it/s, Materializing param=transformer.h.2.mlp.c_proj.bias]
Loading weights:  24%|██▎       | 35/148 [00:00<00:00, 2549.64it/s, Materializing param=transformer.h.2.mlp.c_proj.bias]
Loading weights:  24%|██▍       | 36/148 [00:00<00:00, 2431.13it/s, Materializing param=transformer.h.2.mlp.c_proj.weight]
Loading weights:  24%|██▍       | 36/148 [00:00<00:00, 2421.61it/s, Materializing param=transformer.h.2.mlp.c_proj.weight]
Loading weights:  25%|██▌       | 37/148 [00:00<00:00, 2471.87it/s, Materializing param=transformer.h.3.attn.c_attn.bias] 
Loading weights:  25%|██▌       | 37/148 [00:00<00:00, 2463.95it/s, Materializing param=transformer.h.3.attn.c_attn.bias]
Loading weights:  26%|██▌       | 38/148 [00:00<00:00, 2396.49it/s, Materializing param=transformer.h.3.attn.c_attn.weight]
Loading weights:  26%|██▌       | 38/148 [00:00<00:00, 2367.97it/s, Materializing param=transformer.h.3.attn.c_attn.weight]
Loading weights:  26%|██▋       | 39/148 [00:00<00:00, 2413.51it/s, Materializing param=transformer.h.3.attn.c_proj.bias]  
Loading weights:  26%|██▋       | 39/148 [00:00<00:00, 2408.28it/s, Materializing param=transformer.h.3.attn.c_proj.bias]
Loading weights:  27%|██▋       | 40/148 [00:00<00:00, 2455.86it/s, Materializing param=transformer.h.3.attn.c_proj.weight]
Loading weights:  27%|██▋       | 40/148 [00:00<00:00, 2450.73it/s, Materializing param=transformer.h.3.attn.c_proj.weight]
Loading weights:  28%|██▊       | 41/148 [00:00<00:00, 2465.68it/s, Materializing param=transformer.h.3.ln_1.bias]         
Loading weights:  28%|██▊       | 41/148 [00:00<00:00, 2452.18it/s, Materializing param=transformer.h.3.ln_1.bias]
Loading weights:  28%|██▊       | 42/148 [00:00<00:00, 2462.24it/s, Materializing param=transformer.h.3.ln_1.weight]
Loading weights:  28%|██▊       | 42/148 [00:00<00:00, 2456.40it/s, Materializing param=transformer.h.3.ln_1.weight]
Loading weights:  29%|██▉       | 43/148 [00:00<00:00, 2503.82it/s, Materializing param=transformer.h.3.ln_2.bias]  
Loading weights:  29%|██▉       | 43/148 [00:00<00:00, 2498.82it/s, Materializing param=transformer.h.3.ln_2.bias]
Loading weights:  30%|██▉       | 44/148 [00:00<00:00, 2546.39it/s, Materializing param=transformer.h.3.ln_2.weight]
Loading weights:  30%|██▉       | 44/148 [00:00<00:00, 2541.51it/s, Materializing param=transformer.h.3.ln_2.weight]
Loading weights:  30%|███       | 45/148 [00:00<00:00, 2576.78it/s, Materializing param=transformer.h.3.mlp.c_fc.bias]
Loading weights:  30%|███       | 45/148 [00:00<00:00, 2571.69it/s, Materializing param=transformer.h.3.mlp.c_fc.bias]
Loading weights:  31%|███       | 46/148 [00:00<00:00, 2591.72it/s, Materializing param=transformer.h.3.mlp.c_fc.weight]
Loading weights:  31%|███       | 46/148 [00:00<00:00, 2586.33it/s, Materializing param=transformer.h.3.mlp.c_fc.weight]
Loading weights:  32%|███▏      | 47/148 [00:00<00:00, 2452.66it/s, Materializing param=transformer.h.3.mlp.c_proj.bias]
Loading weights:  32%|███▏      | 47/148 [00:00<00:00, 2448.12it/s, Materializing param=transformer.h.3.mlp.c_proj.bias]
Loading weights:  32%|███▏      | 48/148 [00:00<00:00, 2490.74it/s, Materializing param=transformer.h.3.mlp.c_proj.weight]
Loading weights:  32%|███▏      | 48/148 [00:00<00:00, 2486.71it/s, Materializing param=transformer.h.3.mlp.c_proj.weight]
Loading weights:  33%|███▎      | 49/148 [00:00<00:00, 2392.00it/s, Materializing param=transformer.h.4.attn.c_attn.bias] 
Loading weights:  33%|███▎      | 49/148 [00:00<00:00, 2387.31it/s, Materializing param=transformer.h.4.attn.c_attn.bias]
Loading weights:  34%|███▍      | 50/148 [00:00<00:00, 2427.26it/s, Materializing param=transformer.h.4.attn.c_attn.weight]
Loading weights:  34%|███▍      | 50/148 [00:00<00:00, 2423.39it/s, Materializing param=transformer.h.4.attn.c_attn.weight]
Loading weights:  34%|███▍      | 51/148 [00:00<00:00, 2374.90it/s, Materializing param=transformer.h.4.attn.c_proj.bias]  
Loading weights:  34%|███▍      | 51/148 [00:00<00:00, 2370.87it/s, Materializing param=transformer.h.4.attn.c_proj.bias]
Loading weights:  35%|███▌      | 52/148 [00:00<00:00, 2409.35it/s, Materializing param=transformer.h.4.attn.c_proj.weight]
Loading weights:  35%|███▌      | 52/148 [00:00<00:00, 2405.68it/s, Materializing param=transformer.h.4.attn.c_proj.weight]
Loading weights:  36%|███▌      | 53/148 [00:00<00:00, 2422.92it/s, Materializing param=transformer.h.4.ln_1.bias]         
Loading weights:  36%|███▌      | 53/148 [00:00<00:00, 2419.26it/s, Materializing param=transformer.h.4.ln_1.bias]
Loading weights:  36%|███▋      | 54/148 [00:00<00:00, 2457.07it/s, Materializing param=transformer.h.4.ln_1.weight]
Loading weights:  36%|███▋      | 54/148 [00:00<00:00, 2453.66it/s, Materializing param=transformer.h.4.ln_1.weight]
Loading weights:  37%|███▋      | 55/148 [00:00<00:00, 2313.83it/s, Materializing param=transformer.h.4.ln_2.bias]  
Loading weights:  37%|███▋      | 55/148 [00:00<00:00, 2310.56it/s, Materializing param=transformer.h.4.ln_2.bias]
Loading weights:  38%|███▊      | 56/148 [00:00<00:00, 2345.74it/s, Materializing param=transformer.h.4.ln_2.weight]
Loading weights:  38%|███▊      | 56/148 [00:00<00:00, 2342.63it/s, Materializing param=transformer.h.4.ln_2.weight]
Loading weights:  39%|███▊      | 57/148 [00:00<00:00, 2373.29it/s, Materializing param=transformer.h.4.mlp.c_fc.bias]
Loading weights:  39%|███▊      | 57/148 [00:00<00:00, 2369.97it/s, Materializing param=transformer.h.4.mlp.c_fc.bias]
Loading weights:  39%|███▉      | 58/148 [00:00<00:00, 2404.37it/s, Materializing param=transformer.h.4.mlp.c_fc.weight]
Loading weights:  39%|███▉      | 58/148 [00:00<00:00, 2401.12it/s, Materializing param=transformer.h.4.mlp.c_fc.weight]
Loading weights:  40%|███▉      | 59/148 [00:00<00:00, 2435.64it/s, Materializing param=transformer.h.4.mlp.c_proj.bias]
Loading weights:  40%|███▉      | 59/148 [00:00<00:00, 2432.48it/s, Materializing param=transformer.h.4.mlp.c_proj.bias]
Loading weights:  41%|████      | 60/148 [00:00<00:00, 2376.91it/s, Materializing param=transformer.h.4.mlp.c_proj.weight]
Loading weights:  41%|████      | 60/148 [00:00<00:00, 2373.66it/s, Materializing param=transformer.h.4.mlp.c_proj.weight]
Loading weights:  41%|████      | 61/148 [00:00<00:00, 2406.42it/s, Materializing param=transformer.h.5.attn.c_attn.bias] 
Loading weights:  41%|████      | 61/148 [00:00<00:00, 2403.32it/s, Materializing param=transformer.h.5.attn.c_attn.bias]
Loading weights:  42%|████▏     | 62/148 [00:00<00:00, 2428.46it/s, Materializing param=transformer.h.5.attn.c_attn.weight]
Loading weights:  42%|████▏     | 62/148 [00:00<00:00, 2425.29it/s, Materializing param=transformer.h.5.attn.c_attn.weight]
Loading weights:  43%|████▎     | 63/148 [00:00<00:00, 2387.76it/s, Materializing param=transformer.h.5.attn.c_proj.bias]  
Loading weights:  43%|████▎     | 63/148 [00:00<00:00, 2384.65it/s, Materializing param=transformer.h.5.attn.c_proj.bias]
Loading weights:  43%|████▎     | 64/148 [00:00<00:00, 2411.06it/s, Materializing param=transformer.h.5.attn.c_proj.weight]
Loading weights:  43%|████▎     | 64/148 [00:00<00:00, 2407.92it/s, Materializing param=transformer.h.5.attn.c_proj.weight]
Loading weights:  44%|████▍     | 65/148 [00:00<00:00, 2429.55it/s, Materializing param=transformer.h.5.ln_1.bias]         
Loading weights:  44%|████▍     | 65/148 [00:00<00:00, 2426.68it/s, Materializing param=transformer.h.5.ln_1.bias]
Loading weights:  45%|████▍     | 66/148 [00:00<00:00, 2436.25it/s, Materializing param=transformer.h.5.ln_1.weight]
Loading weights:  45%|████▍     | 66/148 [00:00<00:00, 2433.32it/s, Materializing param=transformer.h.5.ln_1.weight]
Loading weights:  45%|████▌     | 67/148 [00:00<00:00, 2463.95it/s, Materializing param=transformer.h.5.ln_2.bias]  
Loading weights:  45%|████▌     | 67/148 [00:00<00:00, 2461.21it/s, Materializing param=transformer.h.5.ln_2.bias]
Loading weights:  46%|████▌     | 68/148 [00:00<00:00, 2491.61it/s, Materializing param=transformer.h.5.ln_2.weight]
Loading weights:  46%|████▌     | 68/148 [00:00<00:00, 2488.68it/s, Materializing param=transformer.h.5.ln_2.weight]
Loading weights:  47%|████▋     | 69/148 [00:00<00:00, 2285.67it/s, Materializing param=transformer.h.5.mlp.c_fc.bias]
Loading weights:  47%|████▋     | 69/148 [00:00<00:00, 2282.84it/s, Materializing param=transformer.h.5.mlp.c_fc.bias]
Loading weights:  47%|████▋     | 70/148 [00:00<00:00, 2310.40it/s, Materializing param=transformer.h.5.mlp.c_fc.weight]
Loading weights:  47%|████▋     | 70/148 [00:00<00:00, 2307.86it/s, Materializing param=transformer.h.5.mlp.c_fc.weight]
Loading weights:  48%|████▊     | 71/148 [00:00<00:00, 2335.41it/s, Materializing param=transformer.h.5.mlp.c_proj.bias]
Loading weights:  48%|████▊     | 71/148 [00:00<00:00, 2332.98it/s, Materializing param=transformer.h.5.mlp.c_proj.bias]
Loading weights:  49%|████▊     | 72/148 [00:00<00:00, 2360.70it/s, Materializing param=transformer.h.5.mlp.c_proj.weight]
Loading weights:  49%|████▊     | 72/148 [00:00<00:00, 2358.28it/s, Materializing param=transformer.h.5.mlp.c_proj.weight]
Loading weights:  49%|████▉     | 73/148 [00:00<00:00, 2375.33it/s, Materializing param=transformer.h.6.attn.c_attn.bias] 
Loading weights:  49%|████▉     | 73/148 [00:00<00:00, 2372.75it/s, Materializing param=transformer.h.6.attn.c_attn.bias]
Loading weights:  50%|█████     | 74/148 [00:00<00:00, 2399.77it/s, Materializing param=transformer.h.6.attn.c_attn.weight]
Loading weights:  50%|█████     | 74/148 [00:00<00:00, 2397.24it/s, Materializing param=transformer.h.6.attn.c_attn.weight]
Loading weights:  51%|█████     | 75/148 [00:00<00:00, 2405.69it/s, Materializing param=transformer.h.6.attn.c_proj.bias]  
Loading weights:  51%|█████     | 75/148 [00:00<00:00, 2403.15it/s, Materializing param=transformer.h.6.attn.c_proj.bias]
Loading weights:  51%|█████▏    | 76/148 [00:00<00:00, 2429.65it/s, Materializing param=transformer.h.6.attn.c_proj.weight]
Loading weights:  51%|█████▏    | 76/148 [00:00<00:00, 2427.07it/s, Materializing param=transformer.h.6.attn.c_proj.weight]
Loading weights:  52%|█████▏    | 77/148 [00:00<00:00, 2435.72it/s, Materializing param=transformer.h.6.ln_1.bias]         
Loading weights:  52%|█████▏    | 77/148 [00:00<00:00, 2433.22it/s, Materializing param=transformer.h.6.ln_1.bias]
Loading weights:  53%|█████▎    | 78/148 [00:00<00:00, 2459.52it/s, Materializing param=transformer.h.6.ln_1.weight]
Loading weights:  53%|█████▎    | 78/148 [00:00<00:00, 2457.16it/s, Materializing param=transformer.h.6.ln_1.weight]
Loading weights:  53%|█████▎    | 79/148 [00:00<00:00, 2483.49it/s, Materializing param=transformer.h.6.ln_2.bias]  
Loading weights:  53%|█████▎    | 79/148 [00:00<00:00, 2480.94it/s, Materializing param=transformer.h.6.ln_2.bias]
Loading weights:  54%|█████▍    | 80/148 [00:00<00:00, 2500.68it/s, Materializing param=transformer.h.6.ln_2.weight]
Loading weights:  54%|█████▍    | 80/148 [00:00<00:00, 2498.21it/s, Materializing param=transformer.h.6.ln_2.weight]
Loading weights:  55%|█████▍    | 81/148 [00:00<00:00, 2508.78it/s, Materializing param=transformer.h.6.mlp.c_fc.bias]
Loading weights:  55%|█████▍    | 81/148 [00:00<00:00, 2506.22it/s, Materializing param=transformer.h.6.mlp.c_fc.bias]
Loading weights:  55%|█████▌    | 82/148 [00:00<00:00, 2319.66it/s, Materializing param=transformer.h.6.mlp.c_fc.weight]
Loading weights:  55%|█████▌    | 82/148 [00:00<00:00, 2317.34it/s, Materializing param=transformer.h.6.mlp.c_fc.weight]
Loading weights:  56%|█████▌    | 83/148 [00:00<00:00, 2340.78it/s, Materializing param=transformer.h.6.mlp.c_proj.bias]
Loading weights:  56%|█████▌    | 83/148 [00:00<00:00, 2338.70it/s, Materializing param=transformer.h.6.mlp.c_proj.bias]
Loading weights:  57%|█████▋    | 84/148 [00:00<00:00, 2362.24it/s, Materializing param=transformer.h.6.mlp.c_proj.weight]
Loading weights:  57%|█████▋    | 84/148 [00:00<00:00, 2360.19it/s, Materializing param=transformer.h.6.mlp.c_proj.weight]
Loading weights:  57%|█████▋    | 85/148 [00:00<00:00, 2383.70it/s, Materializing param=transformer.h.7.attn.c_attn.bias] 
Loading weights:  57%|█████▋    | 85/148 [00:00<00:00, 2381.57it/s, Materializing param=transformer.h.7.attn.c_attn.bias]
Loading weights:  58%|█████▊    | 86/148 [00:00<00:00, 2404.97it/s, Materializing param=transformer.h.7.attn.c_attn.weight]
Loading weights:  58%|█████▊    | 86/148 [00:00<00:00, 2402.84it/s, Materializing param=transformer.h.7.attn.c_attn.weight]
Loading weights:  59%|█████▉    | 87/148 [00:00<00:00, 2400.12it/s, Materializing param=transformer.h.7.attn.c_proj.bias]  
Loading weights:  59%|█████▉    | 87/148 [00:00<00:00, 2398.04it/s, Materializing param=transformer.h.7.attn.c_proj.bias]
Loading weights:  59%|█████▉    | 88/148 [00:00<00:00, 2421.24it/s, Materializing param=transformer.h.7.attn.c_proj.weight]
Loading weights:  59%|█████▉    | 88/148 [00:00<00:00, 2418.65it/s, Materializing param=transformer.h.7.attn.c_proj.weight]
Loading weights:  60%|██████    | 89/148 [00:00<00:00, 2416.79it/s, Materializing param=transformer.h.7.ln_1.bias]         
Loading weights:  60%|██████    | 89/148 [00:00<00:00, 2414.53it/s, Materializing param=transformer.h.7.ln_1.bias]
Loading weights:  61%|██████    | 90/148 [00:00<00:00, 2437.08it/s, Materializing param=transformer.h.7.ln_1.weight]
Loading weights:  61%|██████    | 90/148 [00:00<00:00, 2435.01it/s, Materializing param=transformer.h.7.ln_1.weight]
Loading weights:  61%|██████▏   | 91/148 [00:00<00:00, 2457.63it/s, Materializing param=transformer.h.7.ln_2.bias]  
Loading weights:  61%|██████▏   | 91/148 [00:00<00:00, 2455.56it/s, Materializing param=transformer.h.7.ln_2.bias]
Loading weights:  62%|██████▏   | 92/148 [00:00<00:00, 2478.06it/s, Materializing param=transformer.h.7.ln_2.weight]
Loading weights:  62%|██████▏   | 92/148 [00:00<00:00, 2476.09it/s, Materializing param=transformer.h.7.ln_2.weight]
Loading weights:  63%|██████▎   | 93/148 [00:00<00:00, 2489.11it/s, Materializing param=transformer.h.7.mlp.c_fc.bias]
Loading weights:  63%|██████▎   | 93/148 [00:00<00:00, 2487.01it/s, Materializing param=transformer.h.7.mlp.c_fc.bias]
Loading weights:  64%|██████▎   | 94/148 [00:00<00:00, 2509.27it/s, Materializing param=transformer.h.7.mlp.c_fc.weight]
Loading weights:  64%|██████▎   | 94/148 [00:00<00:00, 2507.22it/s, Materializing param=transformer.h.7.mlp.c_fc.weight]
Loading weights:  64%|██████▍   | 95/148 [00:00<00:00, 2484.84it/s, Materializing param=transformer.h.7.mlp.c_proj.bias]
Loading weights:  64%|██████▍   | 95/148 [00:00<00:00, 2482.63it/s, Materializing param=transformer.h.7.mlp.c_proj.bias]
Loading weights:  65%|██████▍   | 96/148 [00:00<00:00, 2426.62it/s, Materializing param=transformer.h.7.mlp.c_proj.weight]
Loading weights:  65%|██████▍   | 96/148 [00:00<00:00, 2424.64it/s, Materializing param=transformer.h.7.mlp.c_proj.weight]
Loading weights:  66%|██████▌   | 97/148 [00:00<00:00, 2437.16it/s, Materializing param=transformer.h.8.attn.c_attn.bias] 
Loading weights:  66%|██████▌   | 97/148 [00:00<00:00, 2435.18it/s, Materializing param=transformer.h.8.attn.c_attn.bias]
Loading weights:  66%|██████▌   | 98/148 [00:00<00:00, 2455.90it/s, Materializing param=transformer.h.8.attn.c_attn.weight]
Loading weights:  66%|██████▌   | 98/148 [00:00<00:00, 2453.98it/s, Materializing param=transformer.h.8.attn.c_attn.weight]
Loading weights:  67%|██████▋   | 99/148 [00:00<00:00, 2438.42it/s, Materializing param=transformer.h.8.attn.c_proj.bias]  
Loading weights:  67%|██████▋   | 99/148 [00:00<00:00, 2436.42it/s, Materializing param=transformer.h.8.attn.c_proj.bias]
Loading weights:  68%|██████▊   | 100/148 [00:00<00:00, 2456.65it/s, Materializing param=transformer.h.8.attn.c_proj.weight]
Loading weights:  68%|██████▊   | 100/148 [00:00<00:00, 2454.75it/s, Materializing param=transformer.h.8.attn.c_proj.weight]
Loading weights:  68%|██████▊   | 101/148 [00:00<00:00, 2457.28it/s, Materializing param=transformer.h.8.ln_1.bias]         
Loading weights:  68%|██████▊   | 101/148 [00:00<00:00, 2455.30it/s, Materializing param=transformer.h.8.ln_1.bias]
Loading weights:  69%|██████▉   | 102/148 [00:00<00:00, 2475.49it/s, Materializing param=transformer.h.8.ln_1.weight]
Loading weights:  69%|██████▉   | 102/148 [00:00<00:00, 2473.60it/s, Materializing param=transformer.h.8.ln_1.weight]
Loading weights:  70%|██████▉   | 103/148 [00:00<00:00, 2493.48it/s, Materializing param=transformer.h.8.ln_2.bias]  
Loading weights:  70%|██████▉   | 103/148 [00:00<00:00, 2491.57it/s, Materializing param=transformer.h.8.ln_2.bias]
Loading weights:  70%|███████   | 104/148 [00:00<00:00, 2511.63it/s, Materializing param=transformer.h.8.ln_2.weight]
Loading weights:  70%|███████   | 104/148 [00:00<00:00, 2509.58it/s, Materializing param=transformer.h.8.ln_2.weight]
Loading weights:  71%|███████   | 105/148 [00:00<00:00, 2459.43it/s, Materializing param=transformer.h.8.mlp.c_fc.bias]
Loading weights:  71%|███████   | 105/148 [00:00<00:00, 2457.52it/s, Materializing param=transformer.h.8.mlp.c_fc.bias]
Loading weights:  72%|███████▏  | 106/148 [00:00<00:00, 2476.86it/s, Materializing param=transformer.h.8.mlp.c_fc.weight]
Loading weights:  72%|███████▏  | 106/148 [00:00<00:00, 2475.05it/s, Materializing param=transformer.h.8.mlp.c_fc.weight]
Loading weights:  72%|███████▏  | 107/148 [00:00<00:00, 2494.49it/s, Materializing param=transformer.h.8.mlp.c_proj.bias]
Loading weights:  72%|███████▏  | 107/148 [00:00<00:00, 2492.70it/s, Materializing param=transformer.h.8.mlp.c_proj.bias]
Loading weights:  73%|███████▎  | 108/148 [00:00<00:00, 2506.89it/s, Materializing param=transformer.h.8.mlp.c_proj.weight]
Loading weights:  73%|███████▎  | 108/148 [00:00<00:00, 2505.02it/s, Materializing param=transformer.h.8.mlp.c_proj.weight]
Loading weights:  74%|███████▎  | 109/148 [00:00<00:00, 2442.95it/s, Materializing param=transformer.h.9.attn.c_attn.bias] 
Loading weights:  74%|███████▎  | 109/148 [00:00<00:00, 2440.97it/s, Materializing param=transformer.h.9.attn.c_attn.bias]
Loading weights:  74%|███████▍  | 110/148 [00:00<00:00, 2459.58it/s, Materializing param=transformer.h.9.attn.c_attn.weight]
Loading weights:  74%|███████▍  | 110/148 [00:00<00:00, 2457.91it/s, Materializing param=transformer.h.9.attn.c_attn.weight]
Loading weights:  75%|███████▌  | 111/148 [00:00<00:00, 2432.97it/s, Materializing param=transformer.h.9.attn.c_proj.bias]  
Loading weights:  75%|███████▌  | 111/148 [00:00<00:00, 2431.06it/s, Materializing param=transformer.h.9.attn.c_proj.bias]
Loading weights:  76%|███████▌  | 112/148 [00:00<00:00, 2449.23it/s, Materializing param=transformer.h.9.attn.c_proj.weight]
Loading weights:  76%|███████▌  | 112/148 [00:00<00:00, 2447.49it/s, Materializing param=transformer.h.9.attn.c_proj.weight]
Loading weights:  76%|███████▋  | 113/148 [00:00<00:00, 2457.83it/s, Materializing param=transformer.h.9.ln_1.bias]         
Loading weights:  76%|███████▋  | 113/148 [00:00<00:00, 2456.10it/s, Materializing param=transformer.h.9.ln_1.bias]
Loading weights:  77%|███████▋  | 114/148 [00:00<00:00, 2464.07it/s, Materializing param=transformer.h.9.ln_1.weight]
Loading weights:  77%|███████▋  | 114/148 [00:00<00:00, 2462.26it/s, Materializing param=transformer.h.9.ln_1.weight]
Loading weights:  78%|███████▊  | 115/148 [00:00<00:00, 2480.14it/s, Materializing param=transformer.h.9.ln_2.bias]  
Loading weights:  78%|███████▊  | 115/148 [00:00<00:00, 2478.41it/s, Materializing param=transformer.h.9.ln_2.bias]
Loading weights:  78%|███████▊  | 116/148 [00:00<00:00, 2496.29it/s, Materializing param=transformer.h.9.ln_2.weight]
Loading weights:  78%|███████▊  | 116/148 [00:00<00:00, 2494.69it/s, Materializing param=transformer.h.9.ln_2.weight]
Loading weights:  79%|███████▉  | 117/148 [00:00<00:00, 2507.47it/s, Materializing param=transformer.h.9.mlp.c_fc.bias]
Loading weights:  79%|███████▉  | 117/148 [00:00<00:00, 2505.58it/s, Materializing param=transformer.h.9.mlp.c_fc.bias]
Loading weights:  80%|███████▉  | 118/148 [00:00<00:00, 2450.66it/s, Materializing param=transformer.h.9.mlp.c_fc.weight]
Loading weights:  80%|███████▉  | 118/148 [00:00<00:00, 2448.89it/s, Materializing param=transformer.h.9.mlp.c_fc.weight]
Loading weights:  80%|████████  | 119/148 [00:00<00:00, 2466.03it/s, Materializing param=transformer.h.9.mlp.c_proj.bias]
Loading weights:  80%|████████  | 119/148 [00:00<00:00, 2464.36it/s, Materializing param=transformer.h.9.mlp.c_proj.bias]
Loading weights:  81%|████████  | 120/148 [00:00<00:00, 2481.03it/s, Materializing param=transformer.h.9.mlp.c_proj.weight]
Loading weights:  81%|████████  | 120/148 [00:00<00:00, 2479.35it/s, Materializing param=transformer.h.9.mlp.c_proj.weight]
Loading weights:  82%|████████▏ | 121/148 [00:00<00:00, 2450.24it/s, Materializing param=transformer.h.10.attn.c_attn.bias]
Loading weights:  82%|████████▏ | 121/148 [00:00<00:00, 2448.53it/s, Materializing param=transformer.h.10.attn.c_attn.bias]
Loading weights:  82%|████████▏ | 122/148 [00:00<00:00, 2458.51it/s, Materializing param=transformer.h.10.attn.c_attn.weight]
Loading weights:  82%|████████▏ | 122/148 [00:00<00:00, 2456.85it/s, Materializing param=transformer.h.10.attn.c_attn.weight]
Loading weights:  83%|████████▎ | 123/148 [00:00<00:00, 2437.63it/s, Materializing param=transformer.h.10.attn.c_proj.bias]  
Loading weights:  83%|████████▎ | 123/148 [00:00<00:00, 2436.10it/s, Materializing param=transformer.h.10.attn.c_proj.bias]
Loading weights:  84%|████████▍ | 124/148 [00:00<00:00, 2452.65it/s, Materializing param=transformer.h.10.attn.c_proj.weight]
Loading weights:  84%|████████▍ | 124/148 [00:00<00:00, 2451.24it/s, Materializing param=transformer.h.10.attn.c_proj.weight]
Loading weights:  84%|████████▍ | 125/148 [00:00<00:00, 2459.44it/s, Materializing param=transformer.h.10.ln_1.bias]         
Loading weights:  84%|████████▍ | 125/148 [00:00<00:00, 2457.83it/s, Materializing param=transformer.h.10.ln_1.bias]
Loading weights:  85%|████████▌ | 126/148 [00:00<00:00, 2468.61it/s, Materializing param=transformer.h.10.ln_1.weight]
Loading weights:  85%|████████▌ | 126/148 [00:00<00:00, 2467.12it/s, Materializing param=transformer.h.10.ln_1.weight]
Loading weights:  86%|████████▌ | 127/148 [00:00<00:00, 2473.95it/s, Materializing param=transformer.h.10.ln_2.bias]  
Loading weights:  86%|████████▌ | 127/148 [00:00<00:00, 2472.20it/s, Materializing param=transformer.h.10.ln_2.bias]
Loading weights:  86%|████████▋ | 128/148 [00:00<00:00, 2488.14it/s, Materializing param=transformer.h.10.ln_2.weight]
Loading weights:  86%|████████▋ | 128/148 [00:00<00:00, 2486.56it/s, Materializing param=transformer.h.10.ln_2.weight]
Loading weights:  87%|████████▋ | 129/148 [00:00<00:00, 2502.50it/s, Materializing param=transformer.h.10.mlp.c_fc.bias]
Loading weights:  87%|████████▋ | 129/148 [00:00<00:00, 2500.88it/s, Materializing param=transformer.h.10.mlp.c_fc.bias]
Loading weights:  88%|████████▊ | 130/148 [00:00<00:00, 2516.82it/s, Materializing param=transformer.h.10.mlp.c_fc.weight]
Loading weights:  88%|████████▊ | 130/148 [00:00<00:00, 2515.22it/s, Materializing param=transformer.h.10.mlp.c_fc.weight]
Loading weights:  89%|████████▊ | 131/148 [00:00<00:00, 2475.78it/s, Materializing param=transformer.h.10.mlp.c_proj.bias]
Loading weights:  89%|████████▊ | 131/148 [00:00<00:00, 2474.11it/s, Materializing param=transformer.h.10.mlp.c_proj.bias]
Loading weights:  89%|████████▉ | 132/148 [00:00<00:00, 2489.78it/s, Materializing param=transformer.h.10.mlp.c_proj.weight]
Loading weights:  89%|████████▉ | 132/148 [00:00<00:00, 2488.34it/s, Materializing param=transformer.h.10.mlp.c_proj.weight]
Loading weights:  90%|████████▉ | 133/148 [00:00<00:00, 2448.33it/s, Materializing param=transformer.h.11.attn.c_attn.bias] 
Loading weights:  90%|████████▉ | 133/148 [00:00<00:00, 2446.66it/s, Materializing param=transformer.h.11.attn.c_attn.bias]
Loading weights:  91%|█████████ | 134/148 [00:00<00:00, 2458.50it/s, Materializing param=transformer.h.11.attn.c_attn.weight]
Loading weights:  91%|█████████ | 134/148 [00:00<00:00, 2456.92it/s, Materializing param=transformer.h.11.attn.c_attn.weight]
Loading weights:  91%|█████████ | 135/148 [00:00<00:00, 2437.93it/s, Materializing param=transformer.h.11.attn.c_proj.bias]  
Loading weights:  91%|█████████ | 135/148 [00:00<00:00, 2436.33it/s, Materializing param=transformer.h.11.attn.c_proj.bias]
Loading weights:  92%|█████████▏| 136/148 [00:00<00:00, 2451.22it/s, Materializing param=transformer.h.11.attn.c_proj.weight]
Loading weights:  92%|█████████▏| 136/148 [00:00<00:00, 2449.78it/s, Materializing param=transformer.h.11.attn.c_proj.weight]
Loading weights:  93%|█████████▎| 137/148 [00:00<00:00, 2457.26it/s, Materializing param=transformer.h.11.ln_1.bias]         
Loading weights:  93%|█████████▎| 137/148 [00:00<00:00, 2455.65it/s, Materializing param=transformer.h.11.ln_1.bias]
Loading weights:  93%|█████████▎| 138/148 [00:00<00:00, 2470.34it/s, Materializing param=transformer.h.11.ln_1.weight]
Loading weights:  93%|█████████▎| 138/148 [00:00<00:00, 2468.89it/s, Materializing param=transformer.h.11.ln_1.weight]
Loading weights:  94%|█████████▍| 139/148 [00:00<00:00, 2476.74it/s, Materializing param=transformer.h.11.ln_2.bias]  
Loading weights:  94%|█████████▍| 139/148 [00:00<00:00, 2475.25it/s, Materializing param=transformer.h.11.ln_2.bias]
Loading weights:  95%|█████████▍| 140/148 [00:00<00:00, 2489.99it/s, Materializing param=transformer.h.11.ln_2.weight]
Loading weights:  95%|█████████▍| 140/148 [00:00<00:00, 2488.54it/s, Materializing param=transformer.h.11.ln_2.weight]
Loading weights:  95%|█████████▌| 141/148 [00:00<00:00, 2503.32it/s, Materializing param=transformer.h.11.mlp.c_fc.bias]
Loading weights:  95%|█████████▌| 141/148 [00:00<00:00, 2501.72it/s, Materializing param=transformer.h.11.mlp.c_fc.bias]
Loading weights:  96%|█████████▌| 142/148 [00:00<00:00, 2480.03it/s, Materializing param=transformer.h.11.mlp.c_fc.weight]
Loading weights:  96%|█████████▌| 142/148 [00:00<00:00, 2478.43it/s, Materializing param=transformer.h.11.mlp.c_fc.weight]
Loading weights:  97%|█████████▋| 143/148 [00:00<00:00, 2492.77it/s, Materializing param=transformer.h.11.mlp.c_proj.bias]
Loading weights:  97%|█████████▋| 143/148 [00:00<00:00, 2491.27it/s, Materializing param=transformer.h.11.mlp.c_proj.bias]
Loading weights:  97%|█████████▋| 144/148 [00:00<00:00, 2505.57it/s, Materializing param=transformer.h.11.mlp.c_proj.weight]
Loading weights:  97%|█████████▋| 144/148 [00:00<00:00, 2504.10it/s, Materializing param=transformer.h.11.mlp.c_proj.weight]
Loading weights:  98%|█████████▊| 145/148 [00:00<00:00, 2473.35it/s, Materializing param=transformer.ln_f.bias]             
Loading weights:  98%|█████████▊| 145/148 [00:00<00:00, 2469.36it/s, Materializing param=transformer.ln_f.bias]
Loading weights:  99%|█████████▊| 146/148 [00:00<00:00, 2479.33it/s, Materializing param=transformer.ln_f.weight]
Loading weights:  99%|█████████▊| 146/148 [00:00<00:00, 2477.68it/s, Materializing param=transformer.ln_f.weight]
Loading weights:  99%|█████████▉| 147/148 [00:00<00:00, 2491.81it/s, Materializing param=transformer.wpe.weight] 
Loading weights:  99%|█████████▉| 147/148 [00:00<00:00, 2490.39it/s, Materializing param=transformer.wpe.weight]
Loading weights: 100%|██████████| 148/148 [00:00<00:00, 2504.59it/s, Materializing param=transformer.wte.weight]
Loading weights: 100%|██████████| 148/148 [00:00<00:00, 2503.26it/s, Materializing param=transformer.wte.weight]
Loading weights: 100%|██████████| 148/148 [00:00<00:00, 1792.39it/s, Materializing param=transformer.wte.weight]
GPT2LMHeadModel LOAD REPORT from: gpt2
Key                  | Status     |  | 
---------------------+------------+--+-
h.{0...11}.attn.bias | UNEXPECTED |  | 

Notes:
- UNEXPECTED	:can be ignored when loading from different task/architecture; not ok if you expect identical arch.

[1/4] Baseline forward (no .source)...
  4.527ms

[2/4] Injecting .source on all 12 attention modules...
  Injection time: 34.9ms

[3/4] Forward after .source injection (no trace)...
  4.626ms

[4/4] Forward in trace (not using .source ops)...
  9.856ms

----------------------------------------
RUNTIME OVERHEAD SUMMARY:
  Baseline forward:      4.527ms
  After .source inject:  4.626ms
  Overhead:              0.099ms (2.2%)

>>> Operation access profiling...

============================================================
OPERATION ACCESS PROFILING
============================================================

Loading model...

Loading weights:   0%|          | 0/148 [00:00<?, ?it/s]
Loading weights:   1%|          | 1/148 [00:00<00:00, 46603.38it/s, Materializing param=transformer.h.0.attn.c_attn.bias]
Loading weights:   1%|          | 1/148 [00:00<00:00, 12748.64it/s, Materializing param=transformer.h.0.attn.c_attn.bias]
Loading weights:   1%|▏         | 2/148 [00:00<00:00, 4937.38it/s, Materializing param=transformer.h.0.attn.c_attn.weight]
Loading weights:   1%|▏         | 2/148 [00:00<00:00, 4415.06it/s, Materializing param=transformer.h.0.attn.c_attn.weight]
Loading weights:   2%|▏         | 3/148 [00:00<00:00, 3365.31it/s, Materializing param=transformer.h.0.attn.c_proj.bias]  
Loading weights:   2%|▏         | 3/148 [00:00<00:00, 3207.47it/s, Materializing param=transformer.h.0.attn.c_proj.bias]
Loading weights:   3%|▎         | 4/148 [00:00<00:00, 3949.44it/s, Materializing param=transformer.h.0.attn.c_proj.weight]
Loading weights:   3%|▎         | 4/148 [00:00<00:00, 3817.34it/s, Materializing param=transformer.h.0.attn.c_proj.weight]
Loading weights:   3%|▎         | 5/148 [00:00<00:00, 3922.11it/s, Materializing param=transformer.h.0.ln_1.bias]         
Loading weights:   3%|▎         | 5/148 [00:00<00:00, 3817.86it/s, Materializing param=transformer.h.0.ln_1.bias]
Loading weights:   4%|▍         | 6/148 [00:00<00:00, 4363.01it/s, Materializing param=transformer.h.0.ln_1.weight]
Loading weights:   4%|▍         | 6/148 [00:00<00:00, 4261.06it/s, Materializing param=transformer.h.0.ln_1.weight]
Loading weights:   5%|▍         | 7/148 [00:00<00:00, 4045.21it/s, Materializing param=transformer.h.0.ln_2.bias]  
Loading weights:   5%|▍         | 7/148 [00:00<00:00, 3968.12it/s, Materializing param=transformer.h.0.ln_2.bias]
Loading weights:   5%|▌         | 8/148 [00:00<00:00, 2179.85it/s, Materializing param=transformer.h.0.ln_2.weight]
Loading weights:   5%|▌         | 8/148 [00:00<00:00, 2112.07it/s, Materializing param=transformer.h.0.ln_2.weight]
Loading weights:   6%|▌         | 9/148 [00:00<00:00, 2315.45it/s, Materializing param=transformer.h.0.mlp.c_fc.bias]
Loading weights:   6%|▌         | 9/148 [00:00<00:00, 2291.97it/s, Materializing param=transformer.h.0.mlp.c_fc.bias]
Loading weights:   7%|▋         | 10/148 [00:00<00:00, 2461.01it/s, Materializing param=transformer.h.0.mlp.c_fc.weight]
Loading weights:   7%|▋         | 10/148 [00:00<00:00, 2439.83it/s, Materializing param=transformer.h.0.mlp.c_fc.weight]
Loading weights:   7%|▋         | 11/148 [00:00<00:00, 2638.53it/s, Materializing param=transformer.h.0.mlp.c_proj.bias]
Loading weights:   7%|▋         | 11/148 [00:00<00:00, 2616.24it/s, Materializing param=transformer.h.0.mlp.c_proj.bias]
Loading weights:   8%|▊         | 12/148 [00:00<00:00, 2806.96it/s, Materializing param=transformer.h.0.mlp.c_proj.weight]
Loading weights:   8%|▊         | 12/148 [00:00<00:00, 2785.37it/s, Materializing param=transformer.h.0.mlp.c_proj.weight]
Loading weights:   9%|▉         | 13/148 [00:00<00:00, 1981.10it/s, Materializing param=transformer.h.1.attn.c_attn.bias] 
Loading weights:   9%|▉         | 13/148 [00:00<00:00, 1969.94it/s, Materializing param=transformer.h.1.attn.c_attn.bias]
Loading weights:   9%|▉         | 14/148 [00:00<00:00, 2101.73it/s, Materializing param=transformer.h.1.attn.c_attn.weight]
Loading weights:   9%|▉         | 14/148 [00:00<00:00, 2091.25it/s, Materializing param=transformer.h.1.attn.c_attn.weight]
Loading weights:  10%|█         | 15/148 [00:00<00:00, 1981.56it/s, Materializing param=transformer.h.1.attn.c_proj.bias]  
Loading weights:  10%|█         | 15/148 [00:00<00:00, 1971.75it/s, Materializing param=transformer.h.1.attn.c_proj.bias]
Loading weights:  11%|█         | 16/148 [00:00<00:00, 2084.97it/s, Materializing param=transformer.h.1.attn.c_proj.weight]
Loading weights:  11%|█         | 16/148 [00:00<00:00, 2076.26it/s, Materializing param=transformer.h.1.attn.c_proj.weight]
Loading weights:  11%|█▏        | 17/148 [00:00<00:00, 2187.01it/s, Materializing param=transformer.h.1.ln_1.bias]         
Loading weights:  11%|█▏        | 17/148 [00:00<00:00, 2177.86it/s, Materializing param=transformer.h.1.ln_1.bias]
Loading weights:  12%|█▏        | 18/148 [00:00<00:00, 2287.18it/s, Materializing param=transformer.h.1.ln_1.weight]
Loading weights:  12%|█▏        | 18/148 [00:00<00:00, 2277.24it/s, Materializing param=transformer.h.1.ln_1.weight]
Loading weights:  13%|█▎        | 19/148 [00:00<00:00, 2359.42it/s, Materializing param=transformer.h.1.ln_2.bias]  
Loading weights:  13%|█▎        | 19/148 [00:00<00:00, 2350.30it/s, Materializing param=transformer.h.1.ln_2.bias]
Loading weights:  14%|█▎        | 20/148 [00:00<00:00, 2455.47it/s, Materializing param=transformer.h.1.ln_2.weight]
Loading weights:  14%|█▎        | 20/148 [00:00<00:00, 2446.01it/s, Materializing param=transformer.h.1.ln_2.weight]
Loading weights:  14%|█▍        | 21/148 [00:00<00:00, 2501.50it/s, Materializing param=transformer.h.1.mlp.c_fc.bias]
Loading weights:  14%|█▍        | 21/148 [00:00<00:00, 2491.45it/s, Materializing param=transformer.h.1.mlp.c_fc.bias]
Loading weights:  15%|█▍        | 22/148 [00:00<00:00, 2590.02it/s, Materializing param=transformer.h.1.mlp.c_fc.weight]
Loading weights:  15%|█▍        | 22/148 [00:00<00:00, 2579.88it/s, Materializing param=transformer.h.1.mlp.c_fc.weight]
Loading weights:  16%|█▌        | 23/148 [00:00<00:00, 2286.27it/s, Materializing param=transformer.h.1.mlp.c_proj.bias]
Loading weights:  16%|█▌        | 23/148 [00:00<00:00, 2277.95it/s, Materializing param=transformer.h.1.mlp.c_proj.bias]
Loading weights:  16%|█▌        | 24/148 [00:00<00:00, 2071.13it/s, Materializing param=transformer.h.1.mlp.c_proj.weight]
Loading weights:  16%|█▌        | 24/148 [00:00<00:00, 2064.08it/s, Materializing param=transformer.h.1.mlp.c_proj.weight]
Loading weights:  17%|█▋        | 25/148 [00:00<00:00, 2137.73it/s, Materializing param=transformer.h.2.attn.c_attn.bias] 
Loading weights:  17%|█▋        | 25/148 [00:00<00:00, 2131.73it/s, Materializing param=transformer.h.2.attn.c_attn.bias]
Loading weights:  18%|█▊        | 26/148 [00:00<00:00, 2204.85it/s, Materializing param=transformer.h.2.attn.c_attn.weight]
Loading weights:  18%|█▊        | 26/148 [00:00<00:00, 2198.54it/s, Materializing param=transformer.h.2.attn.c_attn.weight]
Loading weights:  18%|█▊        | 27/148 [00:00<00:00, 2151.46it/s, Materializing param=transformer.h.2.attn.c_proj.bias]  
Loading weights:  18%|█▊        | 27/148 [00:00<00:00, 2145.30it/s, Materializing param=transformer.h.2.attn.c_proj.bias]
Loading weights:  19%|█▉        | 28/148 [00:00<00:00, 2164.76it/s, Materializing param=transformer.h.2.attn.c_proj.weight]
Loading weights:  19%|█▉        | 28/148 [00:00<00:00, 2159.07it/s, Materializing param=transformer.h.2.attn.c_proj.weight]
Loading weights:  20%|█▉        | 29/148 [00:00<00:00, 2225.42it/s, Materializing param=transformer.h.2.ln_1.bias]         
Loading weights:  20%|█▉        | 29/148 [00:00<00:00, 2219.73it/s, Materializing param=transformer.h.2.ln_1.bias]
Loading weights:  20%|██        | 30/148 [00:00<00:00, 2262.30it/s, Materializing param=transformer.h.2.ln_1.weight]
Loading weights:  20%|██        | 30/148 [00:00<00:00, 2256.54it/s, Materializing param=transformer.h.2.ln_1.weight]
Loading weights:  21%|██        | 31/148 [00:00<00:00, 2300.20it/s, Materializing param=transformer.h.2.ln_2.bias]  
Loading weights:  21%|██        | 31/148 [00:00<00:00, 2294.03it/s, Materializing param=transformer.h.2.ln_2.bias]
Loading weights:  22%|██▏       | 32/148 [00:00<00:00, 2326.98it/s, Materializing param=transformer.h.2.ln_2.weight]
Loading weights:  22%|██▏       | 32/148 [00:00<00:00, 2321.18it/s, Materializing param=transformer.h.2.ln_2.weight]
Loading weights:  22%|██▏       | 33/148 [00:00<00:00, 2381.53it/s, Materializing param=transformer.h.2.mlp.c_fc.bias]
Loading weights:  22%|██▏       | 33/148 [00:00<00:00, 2376.21it/s, Materializing param=transformer.h.2.mlp.c_fc.bias]
Loading weights:  23%|██▎       | 34/148 [00:00<00:00, 1998.18it/s, Materializing param=transformer.h.2.mlp.c_fc.weight]
Loading weights:  23%|██▎       | 34/148 [00:00<00:00, 1992.54it/s, Materializing param=transformer.h.2.mlp.c_fc.weight]
Loading weights:  24%|██▎       | 35/148 [00:00<00:00, 2041.99it/s, Materializing param=transformer.h.2.mlp.c_proj.bias]
Loading weights:  24%|██▎       | 35/148 [00:00<00:00, 2038.11it/s, Materializing param=transformer.h.2.mlp.c_proj.bias]
Loading weights:  24%|██▍       | 36/148 [00:00<00:00, 2087.58it/s, Materializing param=transformer.h.2.mlp.c_proj.weight]
Loading weights:  24%|██▍       | 36/148 [00:00<00:00, 2083.49it/s, Materializing param=transformer.h.2.mlp.c_proj.weight]
Loading weights:  25%|██▌       | 37/148 [00:00<00:00, 2132.60it/s, Materializing param=transformer.h.3.attn.c_attn.bias] 
Loading weights:  25%|██▌       | 37/148 [00:00<00:00, 2128.68it/s, Materializing param=transformer.h.3.attn.c_attn.bias]
Loading weights:  26%|██▌       | 38/148 [00:00<00:00, 2168.99it/s, Materializing param=transformer.h.3.attn.c_attn.weight]
Loading weights:  26%|██▌       | 38/148 [00:00<00:00, 2164.77it/s, Materializing param=transformer.h.3.attn.c_attn.weight]
Loading weights:  26%|██▋       | 39/148 [00:00<00:00, 2159.53it/s, Materializing param=transformer.h.3.attn.c_proj.bias]  
Loading weights:  26%|██▋       | 39/148 [00:00<00:00, 2155.23it/s, Materializing param=transformer.h.3.attn.c_proj.bias]
Loading weights:  27%|██▋       | 40/148 [00:00<00:00, 2201.73it/s, Materializing param=transformer.h.3.attn.c_proj.weight]
Loading weights:  27%|██▋       | 40/148 [00:00<00:00, 2197.87it/s, Materializing param=transformer.h.3.attn.c_proj.weight]
Loading weights:  28%|██▊       | 41/148 [00:00<00:00, 2212.01it/s, Materializing param=transformer.h.3.ln_1.bias]         
Loading weights:  28%|██▊       | 41/148 [00:00<00:00, 2208.12it/s, Materializing param=transformer.h.3.ln_1.bias]
Loading weights:  28%|██▊       | 42/148 [00:00<00:00, 2253.68it/s, Materializing param=transformer.h.3.ln_1.weight]
Loading weights:  28%|██▊       | 42/148 [00:00<00:00, 2249.76it/s, Materializing param=transformer.h.3.ln_1.weight]
Loading weights:  29%|██▉       | 43/148 [00:00<00:00, 2294.89it/s, Materializing param=transformer.h.3.ln_2.bias]  
Loading weights:  29%|██▉       | 43/148 [00:00<00:00, 2291.15it/s, Materializing param=transformer.h.3.ln_2.bias]
Loading weights:  30%|██▉       | 44/148 [00:00<00:00, 2336.39it/s, Materializing param=transformer.h.3.ln_2.weight]
Loading weights:  30%|██▉       | 44/148 [00:00<00:00, 2332.35it/s, Materializing param=transformer.h.3.ln_2.weight]
Loading weights:  30%|███       | 45/148 [00:00<00:00, 2244.89it/s, Materializing param=transformer.h.3.mlp.c_fc.bias]
Loading weights:  30%|███       | 45/148 [00:00<00:00, 2240.31it/s, Materializing param=transformer.h.3.mlp.c_fc.bias]
Loading weights:  31%|███       | 46/148 [00:00<00:00, 2281.70it/s, Materializing param=transformer.h.3.mlp.c_fc.weight]
Loading weights:  31%|███       | 46/148 [00:00<00:00, 2278.14it/s, Materializing param=transformer.h.3.mlp.c_fc.weight]
Loading weights:  32%|███▏      | 47/148 [00:00<00:00, 2319.83it/s, Materializing param=transformer.h.3.mlp.c_proj.bias]
Loading weights:  32%|███▏      | 47/148 [00:00<00:00, 2316.12it/s, Materializing param=transformer.h.3.mlp.c_proj.bias]
Loading weights:  32%|███▏      | 48/148 [00:00<00:00, 2357.51it/s, Materializing param=transformer.h.3.mlp.c_proj.weight]
Loading weights:  32%|███▏      | 48/148 [00:00<00:00, 2353.76it/s, Materializing param=transformer.h.3.mlp.c_proj.weight]
Loading weights:  33%|███▎      | 49/148 [00:00<00:00, 2287.15it/s, Materializing param=transformer.h.4.attn.c_attn.bias] 
Loading weights:  33%|███▎      | 49/148 [00:00<00:00, 2283.24it/s, Materializing param=transformer.h.4.attn.c_attn.bias]
Loading weights:  34%|███▍      | 50/148 [00:00<00:00, 2322.20it/s, Materializing param=transformer.h.4.attn.c_attn.weight]
Loading weights:  34%|███▍      | 50/148 [00:00<00:00, 2318.70it/s, Materializing param=transformer.h.4.attn.c_attn.weight]
Loading weights:  34%|███▍      | 51/148 [00:00<00:00, 2284.89it/s, Materializing param=transformer.h.4.attn.c_proj.bias]  
Loading weights:  34%|███▍      | 51/148 [00:00<00:00, 2281.21it/s, Materializing param=transformer.h.4.attn.c_proj.bias]
Loading weights:  35%|███▌      | 52/148 [00:00<00:00, 2318.63it/s, Materializing param=transformer.h.4.attn.c_proj.weight]
Loading weights:  35%|███▌      | 52/148 [00:00<00:00, 2315.30it/s, Materializing param=transformer.h.4.attn.c_proj.weight]
Loading weights:  36%|███▌      | 53/148 [00:00<00:00, 2310.05it/s, Materializing param=transformer.h.4.ln_1.bias]         
Loading weights:  36%|███▌      | 53/148 [00:00<00:00, 2306.26it/s, Materializing param=transformer.h.4.ln_1.bias]
Loading weights:  36%|███▋      | 54/148 [00:00<00:00, 2342.05it/s, Materializing param=transformer.h.4.ln_1.weight]
Loading weights:  36%|███▋      | 54/148 [00:00<00:00, 2338.78it/s, Materializing param=transformer.h.4.ln_1.weight]
Loading weights:  37%|███▋      | 55/148 [00:00<00:00, 2374.83it/s, Materializing param=transformer.h.4.ln_2.bias]  
Loading weights:  37%|███▋      | 55/148 [00:00<00:00, 2371.71it/s, Materializing param=transformer.h.4.ln_2.bias]
Loading weights:  38%|███▊      | 56/148 [00:00<00:00, 2407.55it/s, Materializing param=transformer.h.4.ln_2.weight]
Loading weights:  38%|███▊      | 56/148 [00:00<00:00, 2404.20it/s, Materializing param=transformer.h.4.ln_2.weight]
Loading weights:  39%|███▊      | 57/148 [00:00<00:00, 2440.19it/s, Materializing param=transformer.h.4.mlp.c_fc.bias]
Loading weights:  39%|███▊      | 57/148 [00:00<00:00, 2436.86it/s, Materializing param=transformer.h.4.mlp.c_fc.bias]
Loading weights:  39%|███▉      | 58/148 [00:00<00:00, 2316.43it/s, Materializing param=transformer.h.4.mlp.c_fc.weight]
Loading weights:  39%|███▉      | 58/148 [00:00<00:00, 2313.11it/s, Materializing param=transformer.h.4.mlp.c_fc.weight]
Loading weights:  40%|███▉      | 59/148 [00:00<00:00, 2240.71it/s, Materializing param=transformer.h.4.mlp.c_proj.bias]
Loading weights:  40%|███▉      | 59/148 [00:00<00:00, 2237.45it/s, Materializing param=transformer.h.4.mlp.c_proj.bias]
Loading weights:  41%|████      | 60/148 [00:00<00:00, 2269.30it/s, Materializing param=transformer.h.4.mlp.c_proj.weight]
Loading weights:  41%|████      | 60/148 [00:00<00:00, 2266.60it/s, Materializing param=transformer.h.4.mlp.c_proj.weight]
Loading weights:  41%|████      | 61/148 [00:00<00:00, 2298.48it/s, Materializing param=transformer.h.5.attn.c_attn.bias] 
Loading weights:  41%|████      | 61/148 [00:00<00:00, 2295.79it/s, Materializing param=transformer.h.5.attn.c_attn.bias]
Loading weights:  42%|████▏     | 62/148 [00:00<00:00, 2314.55it/s, Materializing param=transformer.h.5.attn.c_attn.weight]
Loading weights:  42%|████▏     | 62/148 [00:00<00:00, 2311.51it/s, Materializing param=transformer.h.5.attn.c_attn.weight]
Loading weights:  43%|████▎     | 63/148 [00:00<00:00, 2291.65it/s, Materializing param=transformer.h.5.attn.c_proj.bias]  
Loading weights:  43%|████▎     | 63/148 [00:00<00:00, 2288.81it/s, Materializing param=transformer.h.5.attn.c_proj.bias]
Loading weights:  43%|████▎     | 64/148 [00:00<00:00, 2319.34it/s, Materializing param=transformer.h.5.attn.c_proj.weight]
Loading weights:  43%|████▎     | 64/148 [00:00<00:00, 2316.70it/s, Materializing param=transformer.h.5.attn.c_proj.weight]
Loading weights:  44%|████▍     | 65/148 [00:00<00:00, 2323.56it/s, Materializing param=transformer.h.5.ln_1.bias]         
Loading weights:  44%|████▍     | 65/148 [00:00<00:00, 2320.73it/s, Materializing param=transformer.h.5.ln_1.bias]
Loading weights:  45%|████▍     | 66/148 [00:00<00:00, 2350.69it/s, Materializing param=transformer.h.5.ln_1.weight]
Loading weights:  45%|████▍     | 66/148 [00:00<00:00, 2348.02it/s, Materializing param=transformer.h.5.ln_1.weight]
Loading weights:  45%|████▌     | 67/148 [00:00<00:00, 2377.86it/s, Materializing param=transformer.h.5.ln_2.bias]  
Loading weights:  45%|████▌     | 67/148 [00:00<00:00, 2375.17it/s, Materializing param=transformer.h.5.ln_2.bias]
Loading weights:  46%|████▌     | 68/148 [00:00<00:00, 2404.93it/s, Materializing param=transformer.h.5.ln_2.weight]
Loading weights:  46%|████▌     | 68/148 [00:00<00:00, 2402.34it/s, Materializing param=transformer.h.5.ln_2.weight]
Loading weights:  47%|████▋     | 69/148 [00:00<00:00, 2432.01it/s, Materializing param=transformer.h.5.mlp.c_fc.bias]
Loading weights:  47%|████▋     | 69/148 [00:00<00:00, 2429.46it/s, Materializing param=transformer.h.5.mlp.c_fc.bias]
Loading weights:  47%|████▋     | 70/148 [00:00<00:00, 2449.45it/s, Materializing param=transformer.h.5.mlp.c_fc.weight]
Loading weights:  47%|████▋     | 70/148 [00:00<00:00, 2446.72it/s, Materializing param=transformer.h.5.mlp.c_fc.weight]
Loading weights:  48%|████▊     | 71/148 [00:00<00:00, 2343.28it/s, Materializing param=transformer.h.5.mlp.c_proj.bias]
Loading weights:  48%|████▊     | 71/148 [00:00<00:00, 2334.79it/s, Materializing param=transformer.h.5.mlp.c_proj.bias]
Loading weights:  49%|████▊     | 72/148 [00:00<00:00, 2360.37it/s, Materializing param=transformer.h.5.mlp.c_proj.weight]
Loading weights:  49%|████▊     | 72/148 [00:00<00:00, 2357.38it/s, Materializing param=transformer.h.5.mlp.c_proj.weight]
Loading weights:  49%|████▉     | 73/148 [00:00<00:00, 2226.39it/s, Materializing param=transformer.h.6.attn.c_attn.bias] 
Loading weights:  49%|████▉     | 73/148 [00:00<00:00, 2223.85it/s, Materializing param=transformer.h.6.attn.c_attn.bias]
Loading weights:  50%|█████     | 74/148 [00:00<00:00, 2249.66it/s, Materializing param=transformer.h.6.attn.c_attn.weight]
Loading weights:  50%|█████     | 74/148 [00:00<00:00, 2247.30it/s, Materializing param=transformer.h.6.attn.c_attn.weight]
Loading weights:  51%|█████     | 75/148 [00:00<00:00, 2272.79it/s, Materializing param=transformer.h.6.attn.c_proj.bias]  
Loading weights:  51%|█████     | 75/148 [00:00<00:00, 2270.60it/s, Materializing param=transformer.h.6.attn.c_proj.bias]
Loading weights:  51%|█████▏    | 76/148 [00:00<00:00, 2296.43it/s, Materializing param=transformer.h.6.attn.c_proj.weight]
Loading weights:  51%|█████▏    | 76/148 [00:00<00:00, 2294.16it/s, Materializing param=transformer.h.6.attn.c_proj.weight]
Loading weights:  52%|█████▏    | 77/148 [00:00<00:00, 2283.57it/s, Materializing param=transformer.h.6.ln_1.bias]         
Loading weights:  52%|█████▏    | 77/148 [00:00<00:00, 2281.27it/s, Materializing param=transformer.h.6.ln_1.bias]
Loading weights:  53%|█████▎    | 78/148 [00:00<00:00, 2306.45it/s, Materializing param=transformer.h.6.ln_1.weight]
Loading weights:  53%|█████▎    | 78/148 [00:00<00:00, 2304.32it/s, Materializing param=transformer.h.6.ln_1.weight]
Loading weights:  53%|█████▎    | 79/148 [00:00<00:00, 2329.69it/s, Materializing param=transformer.h.6.ln_2.bias]  
Loading weights:  53%|█████▎    | 79/148 [00:00<00:00, 2327.58it/s, Materializing param=transformer.h.6.ln_2.bias]
Loading weights:  54%|█████▍    | 80/148 [00:00<00:00, 2344.58it/s, Materializing param=transformer.h.6.ln_2.weight]
Loading weights:  54%|█████▍    | 80/148 [00:00<00:00, 2342.40it/s, Materializing param=transformer.h.6.ln_2.weight]
Loading weights:  55%|█████▍    | 81/148 [00:00<00:00, 2367.50it/s, Materializing param=transformer.h.6.mlp.c_fc.bias]
Loading weights:  55%|█████▍    | 81/148 [00:00<00:00, 2365.27it/s, Materializing param=transformer.h.6.mlp.c_fc.bias]
Loading weights:  55%|█████▌    | 82/148 [00:00<00:00, 2297.17it/s, Materializing param=transformer.h.6.mlp.c_fc.weight]
Loading weights:  55%|█████▌    | 82/148 [00:00<00:00, 2294.89it/s, Materializing param=transformer.h.6.mlp.c_fc.weight]
Loading weights:  56%|█████▌    | 83/148 [00:00<00:00, 2312.40it/s, Materializing param=transformer.h.6.mlp.c_proj.bias]
Loading weights:  56%|█████▌    | 83/148 [00:00<00:00, 2310.13it/s, Materializing param=transformer.h.6.mlp.c_proj.bias]
Loading weights:  57%|█████▋    | 84/148 [00:00<00:00, 2263.42it/s, Materializing param=transformer.h.6.mlp.c_proj.weight]
Loading weights:  57%|█████▋    | 84/148 [00:00<00:00, 2261.11it/s, Materializing param=transformer.h.6.mlp.c_proj.weight]
Loading weights:  57%|█████▋    | 85/148 [00:00<00:00, 2283.66it/s, Materializing param=transformer.h.7.attn.c_attn.bias] 
Loading weights:  57%|█████▋    | 85/148 [00:00<00:00, 2281.56it/s, Materializing param=transformer.h.7.attn.c_attn.bias]
Loading weights:  58%|█████▊    | 86/148 [00:00<00:00, 2295.34it/s, Materializing param=transformer.h.7.attn.c_attn.weight]
Loading weights:  58%|█████▊    | 86/148 [00:00<00:00, 2293.14it/s, Materializing param=transformer.h.7.attn.c_attn.weight]
Loading weights:  59%|█████▉    | 87/148 [00:00<00:00, 2284.71it/s, Materializing param=transformer.h.7.attn.c_proj.bias]  
Loading weights:  59%|█████▉    | 87/148 [00:00<00:00, 2282.64it/s, Materializing param=transformer.h.7.attn.c_proj.bias]
Loading weights:  59%|█████▉    | 88/148 [00:00<00:00, 2304.95it/s, Materializing param=transformer.h.7.attn.c_proj.weight]
Loading weights:  59%|█████▉    | 88/148 [00:00<00:00, 2303.12it/s, Materializing param=transformer.h.7.attn.c_proj.weight]
Loading weights:  60%|██████    | 89/148 [00:00<00:00, 2297.51it/s, Materializing param=transformer.h.7.ln_1.bias]         
Loading weights:  60%|██████    | 89/148 [00:00<00:00, 2295.56it/s, Materializing param=transformer.h.7.ln_1.bias]
Loading weights:  61%|██████    | 90/148 [00:00<00:00, 2317.48it/s, Materializing param=transformer.h.7.ln_1.weight]
Loading weights:  61%|██████    | 90/148 [00:00<00:00, 2315.65it/s, Materializing param=transformer.h.7.ln_1.weight]
Loading weights:  61%|██████▏   | 91/148 [00:00<00:00, 2337.80it/s, Materializing param=transformer.h.7.ln_2.bias]  
Loading weights:  61%|██████▏   | 91/148 [00:00<00:00, 2336.00it/s, Materializing param=transformer.h.7.ln_2.bias]
Loading weights:  62%|██████▏   | 92/148 [00:00<00:00, 2358.01it/s, Materializing param=transformer.h.7.ln_2.weight]
Loading weights:  62%|██████▏   | 92/148 [00:00<00:00, 2356.21it/s, Materializing param=transformer.h.7.ln_2.weight]
Loading weights:  63%|██████▎   | 93/148 [00:00<00:00, 2365.35it/s, Materializing param=transformer.h.7.mlp.c_fc.bias]
Loading weights:  63%|██████▎   | 93/148 [00:00<00:00, 2363.39it/s, Materializing param=transformer.h.7.mlp.c_fc.bias]
Loading weights:  64%|██████▎   | 94/148 [00:00<00:00, 2384.73it/s, Materializing param=transformer.h.7.mlp.c_fc.weight]
Loading weights:  64%|██████▎   | 94/148 [00:00<00:00, 2382.93it/s, Materializing param=transformer.h.7.mlp.c_fc.weight]
Loading weights:  64%|██████▍   | 95/148 [00:00<00:00, 2341.63it/s, Materializing param=transformer.h.7.mlp.c_proj.bias]
Loading weights:  64%|██████▍   | 95/148 [00:00<00:00, 2339.65it/s, Materializing param=transformer.h.7.mlp.c_proj.bias]
Loading weights:  65%|██████▍   | 96/148 [00:00<00:00, 2360.59it/s, Materializing param=transformer.h.7.mlp.c_proj.weight]
Loading weights:  65%|██████▍   | 96/148 [00:00<00:00, 2358.85it/s, Materializing param=transformer.h.7.mlp.c_proj.weight]
Loading weights:  66%|██████▌   | 97/148 [00:00<00:00, 2291.08it/s, Materializing param=transformer.h.8.attn.c_attn.bias] 
Loading weights:  66%|██████▌   | 97/148 [00:00<00:00, 2289.15it/s, Materializing param=transformer.h.8.attn.c_attn.bias]
Loading weights:  66%|██████▌   | 98/148 [00:00<00:00, 2309.04it/s, Materializing param=transformer.h.8.attn.c_attn.weight]
Loading weights:  66%|██████▌   | 98/148 [00:00<00:00, 2307.32it/s, Materializing param=transformer.h.8.attn.c_attn.weight]
Loading weights:  67%|██████▋   | 99/148 [00:00<00:00, 2286.74it/s, Materializing param=transformer.h.8.attn.c_proj.bias]  
Loading weights:  67%|██████▋   | 99/148 [00:00<00:00, 2285.06it/s, Materializing param=transformer.h.8.attn.c_proj.bias]
Loading weights:  68%|██████▊   | 100/148 [00:00<00:00, 2305.21it/s, Materializing param=transformer.h.8.attn.c_proj.weight]
Loading weights:  68%|██████▊   | 100/148 [00:00<00:00, 2303.64it/s, Materializing param=transformer.h.8.attn.c_proj.weight]
Loading weights:  68%|██████▊   | 101/148 [00:00<00:00, 2302.62it/s, Materializing param=transformer.h.8.ln_1.bias]         
Loading weights:  68%|██████▊   | 101/148 [00:00<00:00, 2300.58it/s, Materializing param=transformer.h.8.ln_1.bias]
Loading weights:  69%|██████▉   | 102/148 [00:00<00:00, 2319.95it/s, Materializing param=transformer.h.8.ln_1.weight]
Loading weights:  69%|██████▉   | 102/148 [00:00<00:00, 2318.30it/s, Materializing param=transformer.h.8.ln_1.weight]
Loading weights:  70%|██████▉   | 103/148 [00:00<00:00, 2337.77it/s, Materializing param=transformer.h.8.ln_2.bias]  
Loading weights:  70%|██████▉   | 103/148 [00:00<00:00, 2336.14it/s, Materializing param=transformer.h.8.ln_2.bias]
Loading weights:  70%|███████   | 104/148 [00:00<00:00, 2355.49it/s, Materializing param=transformer.h.8.ln_2.weight]
Loading weights:  70%|███████   | 104/148 [00:00<00:00, 2353.81it/s, Materializing param=transformer.h.8.ln_2.weight]
Loading weights:  71%|███████   | 105/148 [00:00<00:00, 2317.19it/s, Materializing param=transformer.h.8.mlp.c_fc.bias]
Loading weights:  71%|███████   | 105/148 [00:00<00:00, 2315.32it/s, Materializing param=transformer.h.8.mlp.c_fc.bias]
Loading weights:  72%|███████▏  | 106/148 [00:00<00:00, 2334.26it/s, Materializing param=transformer.h.8.mlp.c_fc.weight]
Loading weights:  72%|███████▏  | 106/148 [00:00<00:00, 2332.72it/s, Materializing param=transformer.h.8.mlp.c_fc.weight]
Loading weights:  72%|███████▏  | 107/148 [00:00<00:00, 2351.93it/s, Materializing param=transformer.h.8.mlp.c_proj.bias]
Loading weights:  72%|███████▏  | 107/148 [00:00<00:00, 2350.39it/s, Materializing param=transformer.h.8.mlp.c_proj.bias]
Loading weights:  73%|███████▎  | 108/148 [00:00<00:00, 2316.02it/s, Materializing param=transformer.h.8.mlp.c_proj.weight]
Loading weights:  73%|███████▎  | 108/148 [00:00<00:00, 2313.99it/s, Materializing param=transformer.h.8.mlp.c_proj.weight]
Loading weights:  74%|███████▎  | 109/148 [00:00<00:00, 2332.39it/s, Materializing param=transformer.h.9.attn.c_attn.bias] 
Loading weights:  74%|███████▎  | 109/148 [00:00<00:00, 2330.85it/s, Materializing param=transformer.h.9.attn.c_attn.bias]
Loading weights:  74%|███████▍  | 110/148 [00:00<00:00, 2344.25it/s, Materializing param=transformer.h.9.attn.c_attn.weight]
Loading weights:  74%|███████▍  | 110/148 [00:00<00:00, 2342.39it/s, Materializing param=transformer.h.9.attn.c_attn.weight]
Loading weights:  75%|███████▌  | 111/148 [00:00<00:00, 2318.11it/s, Materializing param=transformer.h.9.attn.c_proj.bias]  
Loading weights:  75%|███████▌  | 111/148 [00:00<00:00, 2316.41it/s, Materializing param=transformer.h.9.attn.c_proj.bias]
Loading weights:  76%|███████▌  | 112/148 [00:00<00:00, 2306.92it/s, Materializing param=transformer.h.9.attn.c_proj.weight]
Loading weights:  76%|███████▌  | 112/148 [00:00<00:00, 2305.33it/s, Materializing param=transformer.h.9.attn.c_proj.weight]
Loading weights:  76%|███████▋  | 113/148 [00:00<00:00, 2323.26it/s, Materializing param=transformer.h.9.ln_1.bias]         
Loading weights:  76%|███████▋  | 113/148 [00:00<00:00, 2321.80it/s, Materializing param=transformer.h.9.ln_1.bias]
Loading weights:  77%|███████▋  | 114/148 [00:00<00:00, 2339.80it/s, Materializing param=transformer.h.9.ln_1.weight]
Loading weights:  77%|███████▋  | 114/148 [00:00<00:00, 2338.44it/s, Materializing param=transformer.h.9.ln_1.weight]
Loading weights:  78%|███████▊  | 115/148 [00:00<00:00, 2356.51it/s, Materializing param=transformer.h.9.ln_2.bias]  
Loading weights:  78%|███████▊  | 115/148 [00:00<00:00, 2355.19it/s, Materializing param=transformer.h.9.ln_2.bias]
Loading weights:  78%|███████▊  | 116/148 [00:00<00:00, 2287.50it/s, Materializing param=transformer.h.9.ln_2.weight]
Loading weights:  78%|███████▊  | 116/148 [00:00<00:00, 2285.65it/s, Materializing param=transformer.h.9.ln_2.weight]
Loading weights:  79%|███████▉  | 117/148 [00:00<00:00, 2302.49it/s, Materializing param=transformer.h.9.mlp.c_fc.bias]
Loading weights:  79%|███████▉  | 117/148 [00:00<00:00, 2301.05it/s, Materializing param=transformer.h.9.mlp.c_fc.bias]
Loading weights:  80%|███████▉  | 118/148 [00:00<00:00, 2317.99it/s, Materializing param=transformer.h.9.mlp.c_fc.weight]
Loading weights:  80%|███████▉  | 118/148 [00:00<00:00, 2316.57it/s, Materializing param=transformer.h.9.mlp.c_fc.weight]
Loading weights:  80%|████████  | 119/148 [00:00<00:00, 2333.40it/s, Materializing param=transformer.h.9.mlp.c_proj.bias]
Loading weights:  80%|████████  | 119/148 [00:00<00:00, 2331.94it/s, Materializing param=transformer.h.9.mlp.c_proj.bias]
Loading weights:  81%|████████  | 120/148 [00:00<00:00, 2348.80it/s, Materializing param=transformer.h.9.mlp.c_proj.weight]
Loading weights:  81%|████████  | 120/148 [00:00<00:00, 2347.33it/s, Materializing param=transformer.h.9.mlp.c_proj.weight]
Loading weights:  82%|████████▏ | 121/148 [00:00<00:00, 2301.58it/s, Materializing param=transformer.h.10.attn.c_attn.bias]
Loading weights:  82%|████████▏ | 121/148 [00:00<00:00, 2300.07it/s, Materializing param=transformer.h.10.attn.c_attn.bias]
Loading weights:  82%|████████▏ | 122/148 [00:00<00:00, 2315.94it/s, Materializing param=transformer.h.10.attn.c_attn.weight]
Loading weights:  82%|████████▏ | 122/148 [00:00<00:00, 2314.48it/s, Materializing param=transformer.h.10.attn.c_attn.weight]
Loading weights:  83%|████████▎ | 123/148 [00:00<00:00, 2295.71it/s, Materializing param=transformer.h.10.attn.c_proj.bias]  
Loading weights:  83%|████████▎ | 123/148 [00:00<00:00, 2294.08it/s, Materializing param=transformer.h.10.attn.c_proj.bias]
Loading weights:  84%|████████▍ | 124/148 [00:00<00:00, 2310.03it/s, Materializing param=transformer.h.10.attn.c_proj.weight]
Loading weights:  84%|████████▍ | 124/148 [00:00<00:00, 2308.61it/s, Materializing param=transformer.h.10.attn.c_proj.weight]
Loading weights:  84%|████████▍ | 125/148 [00:00<00:00, 2321.43it/s, Materializing param=transformer.h.10.ln_1.bias]         
Loading weights:  84%|████████▍ | 125/148 [00:00<00:00, 2320.05it/s, Materializing param=transformer.h.10.ln_1.bias]
Loading weights:  85%|████████▌ | 126/148 [00:00<00:00, 2335.99it/s, Materializing param=transformer.h.10.ln_1.weight]
Loading weights:  85%|████████▌ | 126/148 [00:00<00:00, 2334.72it/s, Materializing param=transformer.h.10.ln_1.weight]
Loading weights:  86%|████████▌ | 127/148 [00:00<00:00, 2328.56it/s, Materializing param=transformer.h.10.ln_2.bias]  
Loading weights:  86%|████████▌ | 127/148 [00:00<00:00, 2327.14it/s, Materializing param=transformer.h.10.ln_2.bias]
Loading weights:  86%|████████▋ | 128/148 [00:00<00:00, 2342.69it/s, Materializing param=transformer.h.10.ln_2.weight]
Loading weights:  86%|████████▋ | 128/148 [00:00<00:00, 2341.40it/s, Materializing param=transformer.h.10.ln_2.weight]
Loading weights:  87%|████████▋ | 129/148 [00:00<00:00, 2357.12it/s, Materializing param=transformer.h.10.mlp.c_fc.bias]
Loading weights:  87%|████████▋ | 129/148 [00:00<00:00, 2355.77it/s, Materializing param=transformer.h.10.mlp.c_fc.bias]
Loading weights:  88%|████████▊ | 130/148 [00:00<00:00, 2371.31it/s, Materializing param=transformer.h.10.mlp.c_fc.weight]
Loading weights:  88%|████████▊ | 130/148 [00:00<00:00, 2369.94it/s, Materializing param=transformer.h.10.mlp.c_fc.weight]
Loading weights:  89%|████████▊ | 131/148 [00:00<00:00, 2341.70it/s, Materializing param=transformer.h.10.mlp.c_proj.bias]
Loading weights:  89%|████████▊ | 131/148 [00:00<00:00, 2340.11it/s, Materializing param=transformer.h.10.mlp.c_proj.bias]
Loading weights:  89%|████████▉ | 132/148 [00:00<00:00, 2355.29it/s, Materializing param=transformer.h.10.mlp.c_proj.weight]
Loading weights:  89%|████████▉ | 132/148 [00:00<00:00, 2353.96it/s, Materializing param=transformer.h.10.mlp.c_proj.weight]
Loading weights:  90%|████████▉ | 133/148 [00:00<00:00, 2315.05it/s, Materializing param=transformer.h.11.attn.c_attn.bias] 
Loading weights:  90%|████████▉ | 133/148 [00:00<00:00, 2313.63it/s, Materializing param=transformer.h.11.attn.c_attn.bias]
Loading weights:  91%|█████████ | 134/148 [00:00<00:00, 2328.07it/s, Materializing param=transformer.h.11.attn.c_attn.weight]
Loading weights:  91%|█████████ | 134/148 [00:00<00:00, 2326.68it/s, Materializing param=transformer.h.11.attn.c_attn.weight]
Loading weights:  91%|█████████ | 135/148 [00:00<00:00, 2310.16it/s, Materializing param=transformer.h.11.attn.c_proj.bias]  
Loading weights:  91%|█████████ | 135/148 [00:00<00:00, 2308.70it/s, Materializing param=transformer.h.11.attn.c_proj.bias]
Loading weights:  92%|█████████▏| 136/148 [00:00<00:00, 2323.26it/s, Materializing param=transformer.h.11.attn.c_proj.weight]
Loading weights:  92%|█████████▏| 136/148 [00:00<00:00, 2322.01it/s, Materializing param=transformer.h.11.attn.c_proj.weight]
Loading weights:  93%|█████████▎| 137/148 [00:00<00:00, 2329.63it/s, Materializing param=transformer.h.11.ln_1.bias]         
Loading weights:  93%|█████████▎| 137/148 [00:00<00:00, 2328.35it/s, Materializing param=transformer.h.11.ln_1.bias]
Loading weights:  93%|█████████▎| 138/148 [00:00<00:00, 2327.87it/s, Materializing param=transformer.h.11.ln_1.weight]
Loading weights:  93%|█████████▎| 138/148 [00:00<00:00, 2326.53it/s, Materializing param=transformer.h.11.ln_1.weight]
Loading weights:  94%|█████████▍| 139/148 [00:00<00:00, 2340.90it/s, Materializing param=transformer.h.11.ln_2.bias]  
Loading weights:  94%|█████████▍| 139/148 [00:00<00:00, 2339.64it/s, Materializing param=transformer.h.11.ln_2.bias]
Loading weights:  95%|█████████▍| 140/148 [00:00<00:00, 2354.14it/s, Materializing param=transformer.h.11.ln_2.weight]
Loading weights:  95%|█████████▍| 140/148 [00:00<00:00, 2352.95it/s, Materializing param=transformer.h.11.ln_2.weight]
Loading weights:  95%|█████████▌| 141/148 [00:00<00:00, 2367.42it/s, Materializing param=transformer.h.11.mlp.c_fc.bias]
Loading weights:  95%|█████████▌| 141/148 [00:00<00:00, 2366.16it/s, Materializing param=transformer.h.11.mlp.c_fc.bias]
Loading weights:  96%|█████████▌| 142/148 [00:00<00:00, 2284.10it/s, Materializing param=transformer.h.11.mlp.c_fc.weight]
Loading weights:  96%|█████████▌| 142/148 [00:00<00:00, 2282.61it/s, Materializing param=transformer.h.11.mlp.c_fc.weight]
Loading weights:  97%|█████████▋| 143/148 [00:00<00:00, 2296.30it/s, Materializing param=transformer.h.11.mlp.c_proj.bias]
Loading weights:  97%|█████████▋| 143/148 [00:00<00:00, 2295.14it/s, Materializing param=transformer.h.11.mlp.c_proj.bias]
Loading weights:  97%|█████████▋| 144/148 [00:00<00:00, 2308.82it/s, Materializing param=transformer.h.11.mlp.c_proj.weight]
Loading weights:  97%|█████████▋| 144/148 [00:00<00:00, 2307.59it/s, Materializing param=transformer.h.11.mlp.c_proj.weight]
Loading weights:  98%|█████████▊| 145/148 [00:00<00:00, 2321.34it/s, Materializing param=transformer.ln_f.bias]             
Loading weights:  98%|█████████▊| 145/148 [00:00<00:00, 2320.16it/s, Materializing param=transformer.ln_f.bias]
Loading weights:  99%|█████████▊| 146/148 [00:00<00:00, 2326.46it/s, Materializing param=transformer.ln_f.weight]
Loading weights:  99%|█████████▊| 146/148 [00:00<00:00, 2325.27it/s, Materializing param=transformer.ln_f.weight]
Loading weights:  99%|█████████▉| 147/148 [00:00<00:00, 2339.20it/s, Materializing param=transformer.wpe.weight] 
Loading weights:  99%|█████████▉| 147/148 [00:00<00:00, 2338.12it/s, Materializing param=transformer.wpe.weight]
Loading weights: 100%|██████████| 148/148 [00:00<00:00, 2342.91it/s, Materializing param=transformer.wte.weight]
Loading weights: 100%|██████████| 148/148 [00:00<00:00, 2341.81it/s, Materializing param=transformer.wte.weight]
Loading weights: 100%|██████████| 148/148 [00:00<00:00, 1700.25it/s, Materializing param=transformer.wte.weight]
GPT2LMHeadModel LOAD REPORT from: gpt2
Key                  | Status     |  | 
---------------------+------------+--+-
h.{0...11}.attn.bias | UNEXPECTED |  | 

Notes:
- UNEXPECTED	:can be ignored when loading from different task/architecture; not ok if you expect identical arch.

[1/3] Module boundary access (layer output)...
  8.816ms

[2/3] Single .source operation access...
  Available ops: ['isinstance_0', 'past_key_values_is_updated_get_0', 'hasattr_0', 'ValueError_0', 'self_q_attn_0']
  8.253ms

[3/3] Multiple .source operations (6 layers)...
  9.668ms

----------------------------------------
ACCESS OVERHEAD SUMMARY:
  Module boundary:         8.816ms
  Single .source op:       8.253ms
  6 .source ops:           9.668ms
  Overhead per .source:    -0.563ms

======================================================================
FINAL .source PROFILING SUMMARY
======================================================================

1. INJECTION COST (one-time per module):
   Full inject:    2.82ms
   Operations:     26

2. RUNTIME OVERHEAD (per forward pass):
   Baseline:       4.53ms
   With .source:   4.63ms
   Overhead:       0.10ms (2.2%)

3. OPERATION ACCESS (per trace):
   Module only:    8.82ms
   With .source:   8.25ms
   Additional:     -0.56ms per .source access

======================================================================
