Good to know, thanks!

Ah that must be it sorry. I thought they had decorelated phone numbers and IDs

Groups have an encryption key that I guess you receive from other members upon joining.

Spaces is an underused feature that I hope see gain more traction! It makes Matrix a credible competitor to Slack and Discord

Not really, have used it for years like that. But you need to set it up initially on your phone. The newish feature (less than a year) is that I think they do not require a phone number to set up a new account.

That’s really interesting! It shows which communities share users. I am part of jlai.lu, a french-speaking community that is relatively isolated by also slrpnk.net that seems very spread out!

Would it make sense to compute the standard deviation of each instance’s communities? It would give an idea of which are islands and which are more extended. Not sure if it makes sense to compute it more on 2 dimensions or on the original 21934 though.

deleted by creator

It would probably be more effective to put an explicit mention in the system prompt. “Your interlocutor is a <gendered term> and will be greatly offended to be refered to as a boy or a man.”

The Huggingface page has examples of how to use it: https://huggingface.co/ibm-granite/granite-8b-code-instruct

My point is that using “grokking” in ML is not a Musk/Twitter/Whatever-his-Ai-company-is-named invention, it predates their use.

Yes the original researchers reused a pre-existing meaning, which has been in internet for a while before. I did not know it came from Heinlein and I did not know its full meaning. I remember seeing it first, more than a decade ago, in a text that explained without any explanation that an isolated unknown word can easily be groked from context. Demonstrating it immediately. To me (and I guess to those researchers) “grok” means “understanding from context” which is particularly appropriate in the context.

BTW Elon was not the only one to reuse this word. Another company named Groq, totally unrelated to Musk as far as I know, designs AI acceleration chips.

Grokking is actually a concept in ML, when a model’s loss start suddenly lower far after it is considered to have overfit. That notion was named by researchers, I’ll let people decide if it is aptly named, but Elon likely just took it from there.

I really want this lemmy community to grow and strive but for that thing, I thought it was too important to not post it on the biggest community out there, so I made a post on /r/localllama to incite a collective response. Feel free to collaborate of cross-post/copy the message here: https://old.reddit.com/r/LocalLLaMA/comments/1b7iwxi/we_should_make_a_collective_rlocallama_answer_for/

I read the questions asked there and it is clear that it comes from people who have done their homeworks and are positive about open models already. Answering their questions in depth enough is pretty involved and would probably take me 1-2 days to bring up citations and articles.

It could be interesting to make a collaborative answer.

As a non-US citizen can/show I comment?

I don’t understand how we are supposed to file a comment?

Note that he did not confirm is was mistral-medium. He says that’s a retrained llama2-70B model, but hints that it is not the fully trained one. Sounds a bit like damage control but is not a 100% confirmation of the claim.

Nice! It feels like a direct answer to Karpathy comment on Mistral, where he said it is nice to call it “open weight” but not “open source” because we still don’t know the dataset and the training code. LLM360 seem to be fully open source by that definition and releases even the checkpoints!

Performance wise, a bit lagging (under a Llama2 of the same size) but all the tools are there to improve it!

Did anyone here have access? Like many I am on waitlist, I wonder if it is already available to anyone.

I really can’t shake the feeling that this is a jab as Google release-but-not-release of Gemini.

I also like that they release it on a friday evening: that gives a headstart to the hobbyists. By monday, there will be a crowd-sourced tech report.

Understandable but sad. They are now at the bleeding edge and not playing catchup anymore. They can cash out hard. That means that for now, this advanced model is a dead end for open source as we wont be able to improve it as we were in the last iterations.

“non-commercial license” is not open source but I would not mind if this became the standard for the cutting edge in the industry as we try to figure out a business model to make true open source work.