Building on HF

Daniel Fox PRO

FlameF0X

https://flamef0x.github.io

FlameF0X

AI & ML interests

Pre-training text generator. (Brother, im 18) Please don't try to contact me.

Recent Activity

posted an update about 5 hours ago

I did some testing on the scalability of FWKV. It hits a speed bottleneck at 1B due to the T4’s bandwidth limitations. Theoretically, it should match RWKV’s inference speed if the GPU had more bandwidth. So the 1B size is not accurate.

updated a model 1 day ago

FlameF0X/FWKV-TinyStories

updated a collection 1 day ago

FWKV

View all activity

Organizations

posted an update about 5 hours ago

Post

updated a model 1 day ago

FlameF0X/FWKV-TinyStories

Text Generation • Updated 1 day ago

updated 2 collections 1 day ago

FWKV

Collection

3 items • Updated 1 day ago

Agent Collaborations

Collection

17 items • Updated 1 day ago

updated a Space 1 day ago

FWKV Demo

👁

Generate a short story from your prompt

replied to their post 1 day ago

Not yet. I'm still experimenting.
Once I get something that I'm pleased with I'm going to write a blog.

posted an update 1 day ago

Post

Greetings Hugging Face!

I started a new project called **FWKV** (Feed-forward Weighted Key Value, or Floored Weighted Key Value), a RWKV-style LM that uses FFNNs (Feed-Forward Neural Networks) instead of RNN and floor(W·K·V). I'm hoping to make it much more efficient and scalable than RWKV.

So far I have:

- FlameF0X/FWKV-29M — this one is undertrained and doesn't have a Space yet. In the attached image you can see its speed on a T4 compared to models with the same configuration.

The only model that's fully working right now is:
- FlameF0X/FWKV-TinyStories — trained on TinyStories for one epoch. The demo Space is FlameF0X/FWKV-demo.