Simple summarization of Evolving Deeper LLM Thinking (Google DeepMind)
text
Simple summarization of Evolving Deeper LLM Thinking (Google DeepMind)
new_line
new_line
The process starts by posing a question. 
text
The process starts by posing a question. 
new_line
1) The LLM generates initial responses. 
text
1) The LLM generates initial responses. 
new_line
2) These generated responses are evaluated according to specific criteria (program-based checker). 
text
2) These generated responses are evaluated according to specific criteria (program-based checker). 
new_line
3) The LLM critiques the evaluated results. 
text
3) The LLM critiques the evaluated results. 
new_line
4) The LLM refines the responses based on the evaluation, critique, and original responses. 
text
4) The LLM refines the responses based on the evaluation, critique, and original responses. 
new_line
new_line
The refined response is then fed back into step 2). If it meets the criteria, the process ends. Otherwise, the algorithm generates more responses based on the refined ones (with some being discarded, some remaining, and some responses potentially being merged). 
text
The refined response is then fed back into step 2). If it meets the criteria, the process ends. Otherwise, the algorithm generates more responses based on the refined ones (with some being discarded, some remaining, and some responses potentially being merged). 
new_line
new_line
Through this process, it demonstrated excellent performance in complex scheduling problems (travel planning, meeting scheduling, etc.). It's a viable method for finding highly effective solutions in specific scenarios. 
text
Through this process, it demonstrated excellent performance in complex scheduling problems (travel planning, meeting scheduling, etc.). It's a viable method for finding highly effective solutions in specific scenarios. 
new_line
new_line
However, there are two major drawbacks: 
text
However, there are two major drawbacks: 
new_line
🤔 An excessive number of API calls are required. (While the cost might not be very high, it leads to significant latency.)
text
🤔 An excessive number of API calls are required. (While the cost might not be very high, it leads to significant latency.)
new_line
🤔 The evaluator is program-based. (This limits its use as a general method. It could potentially be modified/implemented using LLM as Judge, but that would introduce additional API costs for evaluation.) 
text
🤔 The evaluator is program-based. (This limits its use as a general method. It could potentially be modified/implemented using LLM as Judge, but that would introduce additional API costs for evaluation.) 
new_line
new_line
https://arxiv.org/abs/2501.09891
https://arxiv.org/abs/2501.09891
link
Simple summarization of Evolving Deeper LLM Thinking (Google DeepMind)

The process starts by posing a question. 
1) The LLM generates initial responses. 
2) These generated responses are evaluated according to specific criteria (program-based checker). 
3) The LLM critiques the evaluated results. 
4) The LLM refines the responses based on the evaluation, critique, and original responses. 

The refined response is then fed back into step 2). If it meets the criteria, the process ends. Otherwise, the algorithm generates more responses based on the refined ones (with some being discarded, some remaining, and some responses potentially being merged). 

Through this process, it demonstrated excellent performance in complex scheduling problems (travel planning, meeting scheduling, etc.). It's a viable method for finding highly effective solutions in specific scenarios. 

However, there are two major drawbacks: 
🤔 An excessive number of API calls are required. (While the cost might not be very high, it leads to significant latency.)
🤔 The evaluator is program-based. (This limits its use as a general method. It could potentially be modified/implemented using LLM as Judge, but that would introduce additional API costs for evaluation.) 

https://arxiv.org/abs/2501.09891
👍
chansung theainerd John6666 AtAndDev Aditya0619 KimRina Tonic OfferL karren-semvox sriramvasu