Machine Translation (MT) technology has been actively researched and used within the localisation industry for a number of years now. Despite this, there is still a lot of uncertainty with localisation buyers and suppliers on when, if and how to use MT. Buyers of localisation are often promised the world with MT (reduced costs and quicker turnaround times) but find it hard to really understand the true ROI that can be gained from adopting the technology.

Using MT in a commercial environment to help reduce localisation turnaround time and cost requires an investment. The big question is whether to invest or not?

Is MT right for my translation requirements?

MT is a tool that can be used independently to produce a fully automated translation, or can be combined with professional post-editing to produce ‘human’ quality translations. That being said, it is certainly not a one size fits all solution.

If you have a need to translate large amounts of technical content, user generated content, or any other less business critical content, then you’re probably already looking into MT solutions. But if you mostly translate highly creative / marketing content, then investigating MT is probably not (and shouldn’t be) at the top of your agenda.

The point is that it’s quite easy to identify whether or not you’re a candidate for using MT. The tricky part is how you can effectively evaluate the suitability based upon your business requirements.

Customised MT

As mentioned above, investing in MT comes at a cost – setting out requirements and use cases, defining relevant workflows, building the MT infrastructure, integrating into localisation workflows, on-going evaluation and improvements etc. We aren’t talking about simply calling an API of an openly available MT system.

MT engines can be customised by domain, by client and even by client product. The more focused the data, the better the results will be. The level of customisation (investment) is not the only thing to consider. Exactly how the engines should be used and what content they translate should also be considered. For example, there is no point building an engine customised on engineering content and then evaluating it based on legal content.

Evaluating MT quality and productivity gains

Evaluating translation quality is all about understanding the end use/purpose and setting expectations from the beginning. This needs to be considered when evaluating MT output. If the intended use is to just get a general understanding of a text, then this should be evaluated in a different way to a translation that needs to be error free and fluent. For example, if 8 out of 10 segments were considered adequate (accurate), this would be a success for a gist translation, but a fail for a translation that needed to be 100% accurate (such as marketing material).

At E-ging TI we make use of the TAUS DQF (Dynamic Quality Framework) tools for MT evaluation. Using these tools we can quickly compare and rank various MT engines by evaluating the quality of the ‘raw’ output based on different QA models including adequacy, fluency and error typology scoring, as well as tracking post-editing productivity. Using a combination of these tools and techniques, tweaked to each MT requirement, the performance of MT engines can be evaluated in a much more focused way – putting end use and quality expectations at the forefront.

Business intelligence – benchmarking MT performance 

MT can add some real benefits to the translation process, but it’s important that you work with a provider that fully understands your requirements, and can offer an open book pilot and evaluation programme. Evaluating MT and assessing suitability shouldn’t be a complex and expensive process.

At E-ging TI we believe that MT is more than simply applying technology. A full machine translation service is about understanding the complete content lifecycle and evaluating how MT can be introduced into localisation workflows to solve real business challenges.

Take the guesswork out of the investment and only agree to an MT pilot that will promise business data that can help calculate ROI and help make more informed business decisions.