Question about greedy_search

🤗Transformers
1

Hello. I am trying to greedy_search with T5ModelForConditionalGeneration.

However, it seems I have to provide input_ids as both the input_ids parameter and as part of model_kwargs parameter of the greedy_search.

The input_ids argument of greedy_search acts as the initial decoded state, while input_ids that is supposed to appear in model_kwargs is passed to self (T5) for inference.

I do not see a way to pass two input_ids argument to the greedy_search function. Right now the function complains that I don’t pass input_ids to its internal T5.

How would I go about using the greedy_search function fot T5?

2

Solved by passing encoder_outputs to greedy_seach.

3

Hi seongmin, I am facing the same issue currently but I am not sure what to do. Would you have a code sample to share on how you managed to pass the encoder_outputs to greedy_search? Thank you.

4

Hi Reuben.

While I can’t share the exact code because I don’t own it, here’s the gist of what I did:

model = T5ForConditionalGeneration(...)
encoder_outputs = model.encoder(encoder_input_ids, return_dict=True, output_hidden_states=True)
decoder_input_ids = torch.tensor([t5config.starter_token_id])
generated = model.greedy_search(decoder_input_ids, encoder_outputs=encoder_outputs)
1 Like
5

Hi @seongmin ,

That was immensely helpful thank you so much! Have a great day :smiley: