1. Meta’s LLM for Coding

Meta released the fine-tuned Llama 2 for coding.

There are three versions of the models:

1. Code Llama - The foundational model for coding. This is the model to use if you want to further fine tune the model.

2. Code Llama Python - The Code Llama version which is specific to Python. Python is the go to language for most tasks.

3. Code Llama Instruct - This is the instruction tuned version of Code Llama. It takes natural language instructions and spits out code. This is the go to model for code generation.

All these models are available in 3 parameter sizes as Llama 2 - 7B, 13B and 30B. The 7B model can run on a single GPU. While 30B is the best model it isn’t exactly the best for real time completion. 7B and 13B are better for that purpose.

There is an extra feature in these models - fill in the middle. So, the models can look at either side of your code and fill in the missing lines.

Why does this matter? In the research paper Meta mentions an “Unnatural Code Llama”. They claim that the model can beat GPT-4. There is speculation that this “Unnatural” model is trained on GPT4 output. That is against OpenAI ToS. And that's why the model isn’t released.

First reviews of the model have been great. This might be the first coding model which is capable of running on a consumer hardware and isn’t a toy model. It has some real practical purposes.

This model is free for commercial and research use. Anyone can take the model, improve on it or roll out services. Phind has released a fine tuned model which might even beat GPT-4.

Why would you pay for OpenAI or Co-pilot on Github if you can run and host a free model?

2. AI Music Incubator

Back in April AI Music caught the music world by surprise. An artist called Ghostwriter977 released a track. This track featured AI-generated voices of Drake and Weeknd.

Music companies were caught unawares. The company at the heart of the storm - UMG - asked streaming sites to take down the song. Apple and Spotify took down the songs. Youtube couldn’t.

UMG and Youtube are now collaborating on an AI Music Incubator.

In an accompanying release Youtube lays out the three principles driving this collaboration:

1. AI music is here to stay

2. Youtube will scale and improve the Content ID system

3. Ensuring that artists are paid for AI generated music

Why does this matter? The biggest challenge for AI music is the copyright issue. Who owns the AI generated music? Voices aren’t copyrightable.

Sites like Spotify, Apple Music, etc can remove songs. But it is going to be difficult for Youtube. Youtube removed The Drake and Weeknd song due to a technicality. UMG claimed that the song contained an illegal sample of another song. Samples are fair use. But that is not going to work all the time. That is why Youtube is promising to overhaul the Content ID system.

The larger question though is how can music companies stop the proliferation of AI music. What if an AI song hosted on a TPB-like site goes viral? On a bulletproof host. Music companies cannot take down the site. They have no claim. There is no technicality to work with. They will need to fight a long and overdrawn case so that courts can set clear rules. IANAL but this is going to be an uphill battle.

3. Microsoft’s Hedging

As evident from its recent spending strategy Microsoft is banking on AI solutions. The company is integrating AI features into its products like Azure, GitHub etc.

The company is a major investor in OpenAI. The enterprise version of OpenAI is available only via Microsoft Azure.

It has also partnered up with Meta to offer Llama 2.

There are new reports that the company will offer a version of Databricks. This version will allow Databricks users to use any model. This includes any open source models.

Why does this matter? It looks like Microsoft is realizing that going all-in on OpenAI might not work. Bing AI caught fire due to GPT-4 integration but it hasn’t moved the needle. Bing’s market share remains unchanged.

Now the company is hedging its bets and putting a finger in every possible LLM pie. Closed source, best in class model but costly model (GPT4)? They got it. Open Source and cheaper model? They got it. BYOM (bring your own model)? They are moving towards it.

But, this might hurt OpenAI. It is spending a huge amount to keep the models running. From a consumer perspective GPT4 API is costly. People can then move their non-essential loads to cheaper models.

🗞️10 AI news highlights and interesting reads

1. Fine tuning GPT3.5 is now available with GPT4 coming this fall. There are also new and updated GPT3.5 models. The fine tuning capabilities will be provided by Scale.

2. Before, everything required a specific model. But ChatGPT is doing a lot for a lot of people. Controlling robots? Check. Searching for code and other things? Check. Create summaries of text? Check. Creating a searchable database from your knowledge base? Check. Coding help? Check. ChatGPT can be used as an “anything tool”.

3. Is AI generated stuff copyrightable? Maybe not. The basic tenet argued here is that the computer scientist ran the AI model to generate the artwork. While the AI model created the artwork, the scientist had commissioned the work. Court says the scientist didn’t do any work. So, he cannot hold the copyright. The interesting question is what is enough human work for the artwork for a copyright? What if I change the parameters and temperature settings etc?

4. Fine tuning vs prompts is going to be an ongoing debate. I have added a learning resource below. But Ethan Mollick argues that you don’t need fine tuning. You need better prompts. GPT-4 and other LLMs are trained on massive datasets and know everything which is there to know. So, you can get better results by writing good prompts than feeding it more data.

5. Solve elaborate problems using LLM through “Graph of Thought”. The author claims that this is much closer to the way we think - a mind map style. They claim to have achieved 62% better sorting results compared to Tree of Thought. While also reducing cost by 31%.

6. Both academic journals and news sites are suffering from AI generated content. While AP says GenAI tools shouldn’t replace journalists.

7. Early in my career I worked on legacy systems. Lot of legacy code is in COBOL. Lot of banks still run these legacy systems. It is difficult and thankless tasks to rewrite this stuff in another language. Now IBM is betting on AI to convert code to Java. am using it to convert TradeStation code to python and it has worked okay so far. Still requires some help to clean stuff. But it knows to adjust for differences in data processing for Tradestation vs python.

8. Are the big companies compensating people fairly for using their data? Most probably not. Google can and will collect data to train models but you cannot use the output of those models freely. A controversial take from Verge about Google having its cake and eating it too.

9. SeamlessM4T - a multimodal model from Meta. It can translate and transcribe speech to speech and speech to text.

10. Move over RLHF. You can improve models by training on its own output. Authors call it Reinforced Self-Training or ReST. You take the best outputs and feed it back to the model. This reinforces the best output.

1. A comprehensive look at Vector Databases

2. Short course on Fine Tuning (using Lamini)

3. Fine tuning Llama 2 on your data