After the announcements yesterday, I got a chance to try the new gemini-1.5-flash model from 
text
After the announcements yesterday, I got a chance to try the new gemini-1.5-flash model from 
@goog1e
mention
goog1e
, it is almost as good as gpt-4o on the StaticAnalaysisEval (
text
, it is almost as good as gpt-4o on the StaticAnalaysisEval (
https://huggingface.co/datasets/patched-codes/static-analysis-eval
patched-codes/static-analysis-eval
dataset
resource
https://huggingface.co/datasets/patched-codes/static-analysis-eval
) It is also a bit faster than gpt-4o and much cheaper. 
text
) It is also a bit faster than gpt-4o and much cheaper. 
new_line
new_line
I did run into a recitation flag with an example in the dataset where the api refused to fix the vulnerability and flagged the input as using copyrighted content. This is something you cannot unset even with the safety filters and seems to be an existing bug 
text
I did run into a recitation flag with an example in the dataset where the api refused to fix the vulnerability and flagged the input as using copyrighted content. This is something you cannot unset even with the safety filters and seems to be an existing bug 
https://issuetracker.google.com/issues/331677495
https://issuetracker.google.com/issues/331677495
link
new_line
new_line
But overall you get gpt-4o level performance for 7% the price, we are thinking of making it default in patchwork - 
text
But overall you get gpt-4o level performance for 7% the price, we are thinking of making it default in patchwork - 
https://github.com/patched-codes/patchwork
https://github.com/patched-codes/patchwork
link
 You can use the 
text
 You can use the 
google_api_key
`google_api_key`
inline_code
 and 
text
 and 
model
`model`
inline_code
 options to choose 
text
 options to choose 
gemini-1.5-flash-latest
`gemini-1.5-flash-latest`
inline_code
 to run it with patchwork.
text
 to run it with patchwork.
After the announcements yesterday, I got a chance to try the new gemini-1.5-flash model from @goog1e, it is almost as good as gpt-4o on the StaticAnalaysisEval (https://huggingface.co/datasets/patched-codes/static-analysis-eval) It is also a bit faster than gpt-4o and much cheaper. 

I did run into a recitation flag with an example in the dataset where the api refused to fix the vulnerability and flagged the input as using copyrighted content. This is something you cannot unset even with the safety filters and seems to be an existing bug https://issuetracker.google.com/issues/331677495

But overall you get gpt-4o level performance for 7% the price, we are thinking of making it default in patchwork - https://github.com/patched-codes/patchwork You can use the `google_api_key` and `model` options to choose `gemini-1.5-flash-latest` to run it with patchwork.
🔥
codelion adamm-hf victor alias8797 osanseviero
🚀
codelion adamm-hf alias8797
🤯
codelion adamm-hf alias8797
6340651b388c3fa40f9a5bc0
https://cdn-avatars.huggingface.co/v1/production/uploads/6340651b388c3fa40f9a5bc0/av1C4_S7bHGxAzOu8lOmG.jpeg
Adam Molnar
adamm-hf
user
62f32eab52ad88c930bb3f3b
https://cdn-avatars.huggingface.co/v1/production/uploads/1677134945205-62f32eab52ad88c930bb3f3b.png
Asankhaya Sharma
codelion
user