The global machine learning race has given us many great things: stupidly high GPU and RAM prices, a great pretext to fire workers while companies pretend that we are not in a global recession (AI is making us so productive! Also please do NOT look at my last quarter), and some truly good models. Although we are most likely not getting AGI, as seemingly every major AI company is converging on creating the best video slop website that money can buy, which is totally what would be reasonable to do when you are close to obtaining computer god.Read more...
Last week I went to Vigo to attend the Spanish PyCon, PyConES. I traveled there fully sponsored by Datamaran, the company that I work for. I attended as a speaker, since a colleague and I prepared a talk proposal that got accepted (something something ESG with clustering and LLM reformulation).
Day 1 🌦 The first day had only two turns for workshop tracks. I attended:
Overcoming the One Billion Row Challenge with Python (Jordi Contestí, Kiko Correoso, Ernesto Coloma Rotger) This workshop showed different tools and approaches that can be used in Python to overcome the somewhat known One Billion Rows Challenge.Read more...
Recently I have been working on writing a tokenizer from scratch in Rust. In the process, I wanted to really understand the implementation of some commonly used tokenizers. Fun, I know.
Moses Tokenizer Webpage http://www2.statmt.org/ and the implementation that I’ll talk about tokenizer.perl.
If you already know Moses, you know. Silent nod. For everyone else, Moses is an NLP framework written in Perl, focused on statistical machine translation. It is super easy to install anywhere and everyone loves it because of it.Read more...
Last September I attended virtually the 15th biennial conference of the Association for Machine Translation in the Americas, A.K.A. AMTA 2022. There I presented an adaptation of my master’s thesis in collaboration with my tutor (shameless self-promotion here). Even though at the moment I am not enrolled in any course, he helped me with several revisions and provided the budget for attendance, so I wanted to thank him before I move on.Read more...