📊 Just measured reasoning capabilities 🧠  of Qwen1.5-7B 🇨🇳 in Target Sentiment Analysis (TSA) both for original texts (🇷🇺) and translated in English (🇺🇸), in zero-shot-learning mode.  Here is what I've noticed:
text
📊 Just measured reasoning capabilities 🧠  of Qwen1.5-7B 🇨🇳 in Target Sentiment Analysis (TSA) both for original texts (🇷🇺) and translated in English (🇺🇸), in zero-shot-learning mode.  Here is what I've noticed:
new_line
☑️ 1. Huge gap 📈 with the smaller Qwen1.5 and Qwen2 (1.8B and 1.8B).  Qwen1.5-7B strongly outperforms their "smaller bros" so that case when scale of the model matters.
text
☑️ 1. Huge gap 📈 with the smaller Qwen1.5 and Qwen2 (1.8B and 1.8B).  Qwen1.5-7B strongly outperforms their "smaller bros" so that case when scale of the model matters.
new_line
☑️ 2.  Qwen1.5-7B in english (🇺🇸) behaves similar but slightly underperforming 📉 to the most latest 7B alternatives ... and even including Phi-3-small (3.4B)
text
☑️ 2.  Qwen1.5-7B in english (🇺🇸) behaves similar but slightly underperforming 📉 to the most latest 7B alternatives ... and even including Phi-3-small (3.4B)
new_line
☑️ 3. On texts in (🇷🇺) there is a certain underperforming  📉  gap between the most latest 7B alternatives: F1=34.1, other 7B starts with 40.23.
text
☑️ 3. On texts in (🇷🇺) there is a certain underperforming  📉  gap between the most latest 7B alternatives: F1=34.1, other 7B starts with 40.23.
new_line
new_line
In terms of responses, for non-english texts  (🇷🇺) model answers strict and behaves similar to FlanT5. 
text
In terms of responses, for non-english texts  (🇷🇺) model answers strict and behaves similar to FlanT5. 
new_line
Curious about improvements in Qwen2-7B 🔥
text
Curious about improvements in Qwen2-7B 🔥
new_line
new_line
Model: 
text
Model: 
https://huggingface.co/Qwen/Qwen1.5-7B-Chat
Qwen/Qwen1.5-7B-Chat
model
resource
https://huggingface.co/Qwen/Qwen1.5-7B-Chat
new_line
Benchmark: 
text
Benchmark: 
https://github.com/nicolay-r/RuSentNE-LLM-Benchmark
https://github.com/nicolay-r/RuSentNE-LLM-Benchmark
link
new_line
Dataset: 
text
Dataset: 
https://github.com/dialogue-evaluation/RuSentNE-evaluation
https://github.com/dialogue-evaluation/RuSentNE-evaluation
link
new_line
Related paper: Large Language Models in Targeted Sentiment Analysis (2404.12342)
text
Related paper: Large Language Models in Targeted Sentiment Analysis (2404.12342)
new_line
Collection: 
text
Collection: 
https://huggingface.co/collections/nicolay-r/sentiment-analysis-665ba391e0eba729021ea101
https://huggingface.co/collections/nicolay-r/sentiment-analysis-665ba391e0eba729021ea101
link
📊 Just measured reasoning capabilities 🧠  of Qwen1.5-7B 🇨🇳 in Target Sentiment Analysis (TSA) both for original texts (🇷🇺) and translated in English (🇺🇸), in zero-shot-learning mode.  Here is what I've noticed:
☑️ 1. Huge gap 📈 with the smaller Qwen1.5 and Qwen2 (1.8B and 1.8B).  Qwen1.5-7B strongly outperforms their "smaller bros" so that case when scale of the model matters.
☑️ 2.  Qwen1.5-7B in english (🇺🇸) behaves similar but slightly underperforming 📉 to the most latest 7B alternatives ... and even including Phi-3-small (3.4B)
☑️ 3. On texts in (🇷🇺) there is a certain underperforming  📉  gap between the most latest 7B alternatives: F1=34.1, other 7B starts with 40.23.

In terms of responses, for non-english texts  (🇷🇺) model answers strict and behaves similar to FlanT5. 
Curious about improvements in Qwen2-7B 🔥

Model: https://huggingface.co/Qwen/Qwen1.5-7B-Chat
Benchmark: https://github.com/nicolay-r/RuSentNE-LLM-Benchmark
Dataset: https://github.com/dialogue-evaluation/RuSentNE-evaluation
Related paper: Large Language Models in Targeted Sentiment Analysis (2404.12342)
Collection: https://huggingface.co/collections/nicolay-r/sentiment-analysis-665ba391e0eba729021ea101
🔥
victor
666d8686a119281ee0559838
/avatars/d7fdc6472f7221dd56af8f3267605133.svg
yaw
megoyaw3
user
64e62d11d27a8292c3637f86
https://cdn-avatars.huggingface.co/v1/production/uploads/64e62d11d27a8292c3637f86/aptDeBHpCJxcREj6KPLN1.jpeg
Nicolay Rusnachenko
nicolay-r
user