- Stable Confusion
- Posts
- Friday Morning Highlights
Here are some highlights of the week for your Friday morning.
nanoGPT: Fast & Furious
Andrej Karpathy (@karpathy) created nanoGPT, reproducing GPT-2 in only 300 lines of code. He'll be showing how he built this incredible repo in his course in the next couple of weeks (https://karpathy.ai/zero-to-hero.html).
Didn't tweet nanoGPT yet (quietly getting it to good shape) but it's trending on HN so here it is :) :
github.com/karpathy/nanoG…
Aspires to be simplest, fastest repo for training/finetuning medium-sized GPTs. So far confirmed it reproduced GPT-2 (124M). 2 simple files of ~300 lines— Andrej Karpathy (@karpathy)
7:04 PM • Jan 11, 2023
Uncanny VALL-E
Microsoft Research introduced a new audio-to-text model to the game: 'VALL-E'.
3 second acoustic prompts output autoregressive audio codes.
In other words, your voice can be reproduced from just a 3 second audio recording.
DALL-E generates pixels from text. Now meet its cousin, VALL-E, that generates audio from text @MSFTResearch!
VALL-E’s resemblance to DALL-E v1 and Parti @GoogleAI is striking. Image and audio are both continuous signals, but they can be quantized into discrete tokens.
1/🧵
— Jim Fan (@DrJimFan)
4:21 PM • Jan 6, 2023
You can check out the demo here.
Competition for ChatGPT?
Is that boss music in OpenAI's ears? Maybe, as it looks like DeepMind's very own LLM-based chatbot is on the horizon.
DeepMind might release 'Sparrow', a ChatGPT competitor, as a beta sometime this year, per Time's interview of DeepMind CEO Demis Hassabis.
Apparently, they really want to make sure the model's reinforcement learning-based features are on point, such that Sparrow can even cite sources (something ChatGPT cannot currently do)
Art of the Day
Prompt: 'Spider-man in the jungle' by syd mead, cold color palette, muted colors, detailed, 8k (from lexica.art).
That's all for today!