When I read the KAN paper, I see physicists casually making fun of the uncertainties in MLPs or Neural nets as a whole:
text
When I read the KAN paper, I see physicists casually making fun of the uncertainties in MLPs or Neural nets as a whole:
new_line
new_line
- "The philosophy here is close to the mindset of physicists, who often care more about typical cases rather than worst cases" lol this went hard on NNs
text
- "The philosophy here is close to the mindset of physicists, who often care more about typical cases rather than worst cases" lol this went hard on NNs
new_line
new_line
- "Finite grid size can approximate the function well with a residue rate independent of the dimension, hence beating curse of dimensionality!" haha.
text
- "Finite grid size can approximate the function well with a residue rate independent of the dimension, hence beating curse of dimensionality!" haha.
new_line
new_line
- "Neural scaling laws are the phenomenon where test loss decreases with more model parameters"
text
- "Neural scaling laws are the phenomenon where test loss decreases with more model parameters"
new_line
new_line
- "Our approach, which assumes the existence of smooth Kolmogorov Arnold representations, decomposes the high-dimensional function into several 1D functions"
text
- "Our approach, which assumes the existence of smooth Kolmogorov Arnold representations, decomposes the high-dimensional function into several 1D functions"
new_line
new_line
Key Differences With MLPs:
text
Key Differences With MLPs:
new_line
- Activation Functions: Unlike MLPs that use fixed activation functions at the nodes, KANs utilize learnable activation functions located on the edges between nodes.
text
- Activation Functions: Unlike MLPs that use fixed activation functions at the nodes, KANs utilize learnable activation functions located on the edges between nodes.
new_line
- Weight Parameters: In KANs, traditional linear weight matrices are absent. Instead, each weight parameter is replaced by a learnable univariate function, specifically a spline. 
text
- Weight Parameters: In KANs, traditional linear weight matrices are absent. Instead, each weight parameter is replaced by a learnable univariate function, specifically a spline. 
new_line
- Summation Nodes: Nodes in KANs perform simple summation of incoming signals without applying non-linear transformations.
text
- Summation Nodes: Nodes in KANs perform simple summation of incoming signals without applying non-linear transformations.
new_line
new_line
Advantages Over MLPs:
text
Advantages Over MLPs:
new_line
- Accuracy: achieve higher accuracy with smaller network sizes compared to larger MLPs in tasks like data fitting and solving partial differential equations (PDEs).
text
- Accuracy: achieve higher accuracy with smaller network sizes compared to larger MLPs in tasks like data fitting and solving partial differential equations (PDEs).
new_line
- Interpretability: Due to their unique structure, KANs are more interpretable than MLPs. 
text
- Interpretability: Due to their unique structure, KANs are more interpretable than MLPs. 
new_line
new_line
Technical Innovations:
text
Technical Innovations:
new_line
- Learnable Edges: learnable functions on network edges presents a novel approach to network design, providing greater flexibility in modeling complex relationships in data.
text
- Learnable Edges: learnable functions on network edges presents a novel approach to network design, providing greater flexibility in modeling complex relationships in data.
new_line
- No Linear Weights: elimination of linear weights reduces the parameters, and potentially simplifies the learning process, focusing on the optimization of univariate function representations.
text
- No Linear Weights: elimination of linear weights reduces the parameters, and potentially simplifies the learning process, focusing on the optimization of univariate function representations.
new_line
new_line
Applications and Practical Use:
text
Applications and Practical Use:
new_line
- Scientific Collaboration: KANs have been applied in scientific settings as tools to help discover or rediscover math
text
- Scientific Collaboration: KANs have been applied in scientific settings as tools to help discover or rediscover math
When I read the KAN paper, I see physicists casually making fun of the uncertainties in MLPs or Neural nets as a whole:

- "The philosophy here is close to the mindset of physicists, who often care more about typical cases rather than worst cases" lol this went hard on NNs

- "Finite grid size can approximate the function well with a residue rate independent of the dimension, hence beating curse of dimensionality!" haha.

- "Neural scaling laws are the phenomenon where test loss decreases with more model parameters"

- "Our approach, which assumes the existence of smooth Kolmogorov Arnold representations, decomposes the high-dimensional function into several 1D functions"

Key Differences With MLPs:
- Activation Functions: Unlike MLPs that use fixed activation functions at the nodes, KANs utilize learnable activation functions located on the edges between nodes.
- Weight Parameters: In KANs, traditional linear weight matrices are absent. Instead, each weight parameter is replaced by a learnable univariate function, specifically a spline. 
- Summation Nodes: Nodes in KANs perform simple summation of incoming signals without applying non-linear transformations.

Advantages Over MLPs:
- Accuracy: achieve higher accuracy with smaller network sizes compared to larger MLPs in tasks like data fitting and solving partial differential equations (PDEs).
- Interpretability: Due to their unique structure, KANs are more interpretable than MLPs. 

Technical Innovations:
- Learnable Edges: learnable functions on network edges presents a novel approach to network design, providing greater flexibility in modeling complex relationships in data.
- No Linear Weights: elimination of linear weights reduces the parameters, and potentially simplifies the learning process, focusing on the optimization of univariate function representations.

Applications and Practical Use:
- Scientific Collaboration: KANs have been applied in scientific settings as tools to help discover or rediscover math
🔥
eramax damerajee Benyamin369 Ramikan-BR
❤️
gsarti thomwolf
🚀
Ramikan-BR
6438a9027de34e8ea7e4b257
https://cdn-avatars.huggingface.co/v1/production/uploads/6438a9027de34e8ea7e4b257/vib8QSd1AWMr_bR9ig_xJ.jpeg
Jaward Sesay
Jaward
user