pere
/

roberta-debug-8

Model card Files Files and versions

roberta-debug-8 / copy_tokenizer_config.py

pere's picture

first submit

a4a3924 over 3 years ago

History Blame Contribute Delete

312 Bytes

	# Code used for copying tokenizer and config

	from transformers import XLMRobertaTokenizerFast, XLMRobertaConfig

	tokenizer = XLMRobertaTokenizerFast.from_pretrained("xlm-roberta-base")
	config = XLMRobertaConfig.from_pretrained("xlm-roberta-base")

	tokenizer.save_pretrained("./")
	config.save_pretrained("./")