LLM Algorithm Engineer2023 - 2024
Cryptography LLM Finetuning
PyTorchLoRADeepspeedTransformers
Summary
Built domain corpora and instruction sets for cryptography, delivering finetuning and evaluation baselines.
Problem
General LLMs were weak in cryptography, with high error rates on specialized reasoning.
Solution
Curated domain data and instructions, applied LoRA finetuning, and built evaluation benchmarks.
Impact
- Improved accuracy and consistency in crypto tasks.
- Established reusable finetuning and evaluation workflows.
Pipeline
Data curation -> instruction design -> LoRA finetune -> evaluation regressions.
Key Techniques
Key Techniques
Domain data cleaning, instruction templates, alignment evaluation, error analysis.