Google

A practical challenge in SFT involves balancing the ideal of creating large, high-quality datasets with the necessity of minimizing the significant costs associated with data construction and computation.

Economic Trade-offs in SFT Data Development

A FinTech startup with a limited budget is developing a chatbot to provide preliminary financial advice. They need to create a dataset to fine-tune a base language model. Evaluate the two strategies below and recommend one. Justify your recommendation by analyzing the trade-offs between data cost, data quality, and the potential risks for this specific application.

SFT Data Strategy for a FinTech Startup

A medical research institute with a limited budget is developing an instruction-following dataset to fine-tune a language model for generating patient-friendly summaries of clinical trial results. Which of the following strategies represents the most sound evaluation of the economic trade-offs involved?

A development team is fine-tuning a language model for a specialized legal domain. They have a fixed budget and must choose between two data creation strategies:

1. Commissioning a small, highly-curated dataset (approx. 1,000 examples) created by legal experts.
2. Generating a much larger dataset (approx. 50,000 examples) using a combination of automated methods and review by non-expert crowd-workers.

Analyze the potential risks and benefits of each strategy, focusing on the trade-offs between data quality, data quantity, and overall project cost. Conclude with a justified recommendation for which strategy the team should pursue.

Learn Before

Related