XRS’s Logs
Categories
All (2)

Just a place where I share my learning notes and progress! Most of my posts are ongoing, so they will be gradually updated.

I stream daily on twitch: twitch.tv/xrsrke

Say hello? Feel free to get in touch at neuralink#7014 in discord, twitter/@xariusrke or hello@xrs.wtf 😊



FSDL: Breaking down parallelism in Megatron-LM

Megatron-LM combines three kinds of parallelism techniques to make the most of the available computational resources.

🧠 The Brain

There are problems in deep learning that we don’t know how to solve (catastrophic forgeting, adversarial attack…). It makes sense to pay some attention to how the brain solves them

Apr 1, 2023
No matching items