My intuition:

• There’re “genuine” instances of hapax legomena which probably have some semantic sense, e.g. a rare concept, a wordplay, an artistic invention, an ancient inside joke.
• There’s various noise because somebody let their cat on the keyboard, because OCR software failed in one small spot, because somebody was copying data using a noisy channel without error correction, because somebody had a headache and couldn’t be bothered, because whatever.
• Once a dataset is too big to be manually reviewed by experts, the amount of general noise is far far far larger than what you’re looking for. At the same time you can’t differentiate between the two using statistics alone. And if it was manually reviewed, the experts have probably published their findings, or at least told a few colleagues.
• Transformers are VERY data-hungry. They need enormous datasets.

So I don’t think this approach will help you a lot even for finding words and phrases. And everything I’ve said can be extended to semantic noise too, so your extended question also seems a hopeless endeavour when approached specifically with LLMs or big data analysis of text.

LLaMA can’t. Chameleon and similar ones can:

• https://arxiv.org/abs/2405.09818

For Tolkien’s work, there is the twelve volume “The Complete History of Middle Earth” which is about as inside baseball as you can get for Tolkien.

I’d replace HoME with Parma Eldalamberon, Vinyar Tengwar and other journals publishing his early materials here.

Recommending Italo Calvino’s Six Memos for the Next Millennium, the lectures he has been preparing shortly before his death.

Not an assembly guide for a work of literature, but it’ll help your own process if it’s already ongoing and you want to improve.

The lectures also have some comments on what Calvino himself was doing here and there and why.

For me specifically, if spoilers hurt a book, it probably wasn’t worth reading in the first place. I love when authors demonstrate mastery of language and narration, and no amount of spoilers can overshadow the direct experience of witnessing it enacted.

ChatMusician isn’t exactly new and the underlying dataset isn’t particularly diverse, but it’s one of the few models made specifically for classical music.

Are there any others, by the way?

I wonder how much Beckett was inspired by this while writing Rough for Theatre II:

B: [Hurriedly.] ‘… morbidly sensitive to the opinion of others at the time, I mean as often and for as long as they entered my awareness–’ What kind of Chinese is that? A: [Nervously.] Keep going, keep going! B: ‘… for as long as they entered my awareness, and that in either case, I mean whether such on the one hand as to give me pleasure or on the contrary on the other to cause me pain, and truth to tell–’ Shit! Where’s the verb? A: What verb? B: The main! A: I give up. B: Hold on till I find the verb and to hell with all this drivel in the middle. [Reading.] ‘… were I but … could I but …’ –Jesus!–‘… though it be … be it but…’–Christ!–ah! I have it–‘… I was unfortunately incapable …’ Done it! A: How does it run now? B: [Solemnly.] ‘… morbidly sensitive to the opinion of others at the time …’–drivel drivel drivel–‘… I was unfortunately incapable–’ [The lamp goes out. Long pause.]

The Phoebus cartel strikes again!

I expected that recording would be the hard part.

I think some of the open-source ones should work if your phone is rooted?

I’ve heard that Google’s phone app can record calls (though it says it aloud when starting the recording). Of course, it wouldn’t work if Google thinks it shouldn’t in your region.

By the way, Bluetooth headphones can have both speakers and a microphone. And Android can’t tell a peripheral device what it should or shouldn’t do with audio streams. Sounds like a fun DIY project if you’re into it, or maybe somebody sells these already.

Haven’t heard of all-in-one solutions, but once you have a recording, whisper.cpp can do the transcription:

• https://github.com/ggerganov/whisper.cpp

The underlying Whisper models are MIT.

Then you can use any LLM inference engine, e.g. llama.cpp, and ask the model of your choice to summarise the transcript:

• https://github.com/ggerganov/llama.cpp

You can also write a small bash/python script to make the process a bit more automatic.

I enjoy xenharmonic music and modern academic music the most, but I’m not familiar with everything there, so any recommendations are welcome if you, reader, have something in your mind.

Because we have tons of ground-level sensors, but not a lot in the upper layers of the atmosphere, I think?

Why is this important? Weather processes are usually modelled as a set of differential equations, and you want to know the border conditions in order to solve them and obtain the state of the entire atmosphere. The atmosphere has two boundaries: the lower, which is the planet’s surface, and the upper, which is where the atmosphere ends. And since we don’t seem to have a lot of data from the upper layers, it reduces the quality of all predictions.

Yeah, while tripping on acid.

Interessen-Gemeinschaft Matte.

Given the fact that there was an unintentional DDOS when federated Lemmy instances were requesting the same preview around the same time, it must be one of LW’s servers, not anything on your side.

The only sure way to get rid of this effect is to use an instance entirely hosted on servers in anglophone countries, I think.

I know Google likes to localise their websites based on IP addresses. Perhaps the preview was requested from a Russian IP? (not necessarily yours, could be a VPN if you use one or one of LW’s servers)

This is not to say that Jung wasn’t a genius. Jung was THE BOMB DIGGIDITY (which, by the way, I wish was an official term in the Oxford dictionary).

If they love Jung so much (which I agree they should because Jung was amaaaaazing), why don’t they honor him by using the spelling he actually used?

Love etymological articles with unreliable narrators.

“These hills are being bombed”?

No IPA notation? ⸨I’m somewhat disappointed⸩

It would. But it’s a good option when you have computationally heavy tasks and communication is relatively light.