pruizf/camembert-base-ft-AS13_stgdir-100

This model is a fine-tuned version of camembert-base on the dataset described below. It achieves the following results on the evaluation set:

• Loss: 0.5500
• Accuracy: 0.8837

Model description

Fine-tuned for stage direction classification in French, using the dataset at https://nakala.fr/10.34847/nkl.fde37ug3.

The categorization scheme and rationale are described in the following publication:

Schneider, Alexia., & Ruiz Fabo, Pablo. (2024). Stage direction classification in French theater: Transfer learning experiments. In Proceedings of the 8th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature (LaTeCH-CLfL 2024) (pp. 278–286). Association for Computational Linguistics. https://aclanthology.org/2024.latechclfl-1.28/

Intended uses & limitations

Stage direction classification in French.

Training and evaluation data

Stage direction dataset annotated with 13 categories by Alexia Schneider & Pablo Ruiz.

The categories were derived from those available at FreDraCor (and originally in the Théâtre Classique platform).

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 10

Training results

On held-out data:

Label	Precision	Recall	F1-score	Support
action	0.8924	0.8704	0.8812	486
aggression	0.7467	0.7467	0.7467	75
aparte	0.0000	0.0000	0.0000	14
delivery	0.8676	0.8310	0.8489	213
entrance	0.7500	0.8672	0.8043	128
exit	0.8797	0.8760	0.8778	242
interaction	0.7377	0.8824	0.8036	102
movement	0.7778	0.6471	0.7064	119
music	0.9775	0.9775	0.9775	577
narration	0.7881	0.7750	0.7815	120
object	0.8398	0.8317	0.8357	208
setting	0.8535	0.8895	0.8711	190
toward	0.9653	0.9911	0.9780	449
Accuracy	—	—	0.8861	2923
Macro avg	0.7751	0.7835	0.7779	2923
Weighted avg	0.8827	0.8861	0.8836	2923

Training details:

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.5573	1.0	585	0.7915	0.8276
0.7079	2.0	1170	0.5399	0.8691
0.4625	3.0	1755	0.4973	0.8734
0.3416	4.0	2340	0.4803	0.8794
0.2643	5.0	2925	0.5116	0.8837
0.1956	6.0	3510	0.5207	0.8824
0.1684	7.0	4095	0.5500	0.8837

Framework versions

• Transformers 4.57.2
• Pytorch 2.9.0+cu126
• Datasets 4.0.0
• Tokenizers 0.22.1