crslab.model.conversation.gpt2 package

Submodules

GPT2

References

Radford, Alec, et al. “Language Models are Unsupervised Multitask Learners.”.

class crslab.model.conversation.gpt2.gpt2.GPT2Model(opt, device, vocab, side_data)[source]

Bases: crslab.model.base.BaseModel

context_truncate

A integer indicating the length of dialogue context.

response_truncate

A integer indicating the length of dialogue response.

pad_id

A integer indicating the id of padding token.

Parameters

  • opt (dict) – A dictionary record the hyper parameters.

  • device (torch.device) – A variable indicating which device to place the data and model.

  • vocab (dict) – A dictionary record the vocabulary information.

  • side_data (dict) – A dictionary record the side data.

build_model()[source]

build model

calculate_loss(logit, labels)[source]
Parameters

  • preds – torch.FloatTensor, shape=(bs, response_truncate, vocab_size)

  • labels – torch.LongTensor, shape=(bs, response_truncate)

forward(batch, mode)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

generate(context)[source]
Parameters

context – torch.tensor, shape=(bs, context_turncate)

Returns

torch.tensor, shape=(bs, reponse_turncate-1)

Return type

generated_response

generate_bs(context, beam=4)[source]

Module contents