Hugging Face's logo
AbstractPhila's picture
🤝 Open to Collab

AbstractPhila PRO

AbstractPhil
AbstractPowered
19 6 30
Sarah54321's profile picture whoashish115's profile picture Quazim0t0's profile picture
·
https://civitai.com/user/AbstractPhila

AI & ML interests

datasets, research papers, experimentation, vision, classification, text encoders, tokenization, llms, diffusion, distillation, and more.

Recent Activity

repliedto their post about 11 hours ago
I approach the next stage with a potential fix for shared global context with local dynamic positional capacity. This involves a fixed position global rotary to represent the local dynamic offset rotary merged from independent components. Experimentally and empirically, the framing system now supports differential framing. This process means the core primary engineering limiter - merging, can properly stabilize, and guarantee a merged functional system does not lose that very learned capacity. Think of this like, a lora that can be merged, providing that optimized learned structure, will preserve an effective structure post-merge and can continue to learn along a full finetune. The earlier experiments needed something called a quiet loss, which used KL and micro batching to sample each system meant to be quiet while another learned. This has a large overhead cost and I've been planning a fix for this for some time. The arms showed this has effective preservation, at a compounding overhead. The solution here, is allocated aleph addressing space on the rotary planar. This experiment spans further back to a lensing experiment I ran some time ago, which enabled models to rotate away from latent space, modify the latent information in that space, then rotate back to the original space. The concept ran like a slug on a saltbed, so it was mothballed, but it did work. This new concept is similar to that, except we do not rotate back. The rotary behavior exists as a direct positional offset, and with those offsets we anchor to the attention present within each component's offset. With this we form a multi-spectrum fixed position offset using the dynamic positioning from the model itself. This will enable deep thought integration of each component without overhead cost. This system here, this solution, will provide full MOE dynamic extension rather than partial, without the AMOE arm overhead. Meaning, this engineering solution will solve current limits with beatrix.
posted an update about 11 hours ago
I approach the next stage with a potential fix for shared global context with local dynamic positional capacity. This involves a fixed position global rotary to represent the local dynamic offset rotary merged from independent components. Experimentally and empirically, the framing system now supports differential framing. This process means the core primary engineering limiter - merging, can properly stabilize, and guarantee a merged functional system does not lose that very learned capacity. Think of this like, a lora that can be merged, providing that optimized learned structure, will preserve an effective structure post-merge and can continue to learn along a full finetune. The earlier experiments needed something called a quiet loss, which used KL and micro batching to sample each system meant to be quiet while another learned. This has a large overhead cost and I've been planning a fix for this for some time. The arms showed this has effective preservation, at a compounding overhead. The solution here, is allocated aleph addressing space on the rotary planar. This experiment spans further back to a lensing experiment I ran some time ago, which enabled models to rotate away from latent space, modify the latent information in that space, then rotate back to the original space. The concept ran like a slug on a saltbed, so it was mothballed, but it did work. This new concept is similar to that, except we do not rotate back. The rotary behavior exists as a direct positional offset, and with those offsets we anchor to the attention present within each component's offset. With this we form a multi-spectrum fixed position offset using the dynamic positioning from the model itself. This will enable deep thought integration of each component without overhead cost. This system here, this solution, will provide full MOE dynamic extension rather than partial, without the AMOE arm overhead. Meaning, this engineering solution will solve current limits with beatrix.
repliedto their post 1 day ago
nano-castor and nano-pollux - (4 variants total) are training complete for the story versions. https://huggingface.co/AbstractPhil/alephllm-nano-castor-pollux Trained with an identical dataset until framing as the mini-beatrix-2s variants, and finetuned for context and framing as an expected behavior. The context windows and framing show the models can capture meaningful storage, read, copy, and recall capacity at 4096 byte depth. They aren't particularly smart, but they are functional, and even capable at driving Anima sliders in a meaningfully similar way as the mini-beatrix-3 variant can. https://huggingface.co/AbstractPhil/alephllm-nano-castor-pollux/blob/main/nano-pollux-dual-v1.1-mx-cpx2/v1_boundaries.json Anima sliders often do not have a bi-directional capacity, so the multi-slider arrays were created. Thus allowing the model to rebound naturally as expected when necessary, but this still isn't strong enough. Up next is the full frame trained variant of the champion; nano-pollux-dual-hub and nano-castor. Both of them are the remaining champions. Neither canon models showed similar capacity as the dual-hub's capacity yielded, so we have our champions. Before continuing to the micro variant, these two will be retrained and issued a full education in art. Everything from color theory, to structural awareness, to semantic relations, techniques, and everything between. Our little nano twins here, will be a cooperative pair meant for the first diffusion edit experiments. There were quite a few experiments, many yielded interesting potentials and information. The most powerful variant being the beatrix 3 as a conditioner, yet the nano variants showed a similarly powerful control. https://huggingface.co/AbstractPhil/geolip-beatrix-anima/blob/main/experiments/e071_anima_setting_relay/sheet_1.jpg Even just untrained regularization training on Anima shows yield. The model is pretty underfitted, which is both helpful and not helpful for this train.
View all activity

Organizations

DeepGHS's profile picture Blog-explorers's profile picture BangumiBase's profile picture Abstract Powered Research's profile picture