🔄 In a Training Loop

Andy Chen

andynoodles

·

yi-hsiang-chen-tw

AI & ML interests

Feel free to contact me through Linkedin

Recent Activity

upvoted a paper 5 days ago

Jina-OCR-v1: Efficient Document Parsing with Speculative Decoding and Dense Verifiable Rewards

repliedto onekq's post 10 days ago

My take on device-side inference: it's all about high bandwidth memory (thinking about it, this holds for the cloud too). MacBooks enjoy incidental capacity of apple silicon, but per-device RAM is too low (16 to 24GB), only sufficient for a decent SLM. 512GB is the highest you can go (Kimi K2*). Counting MLX downloads of Kimi K2* on Huggingface, I estimate the user base to be <25K. On the other hand, the newly debuted DGX station (Nvidia) has 748GB, which can fit in the latest Kimi, DS, and Qwen. Also the quantization options of CUDA is way better than MLX. For high-end inferencing, I place my bet on workstations over Macs.

liked a model 10 days ago

zai-org/GLM-5.3

View all activity

Organizations

upvoted a paper 5 days ago

Jina-OCR-v1: Efficient Document Parsing with Speculative Decoding and Dense Verifiable Rewards

Paper • 2609.03181 • Published 10 days ago • 3