Jul 5
Like I am sorry if this sounds offensive, but if you built a fancy RAG pipeline you are still a user. Even if you managed to host it locally
7
285
building tools for fraud, plagiarism, and cognitive offloading @trymirai
Joined October 2015
- Posts 45,650
- Following 1,193
- Followers 4,383
- Likes 60,595
3,004 Photos and videos
Jul 5
When I was growing up, it was customary for non-tech people to put a “confident PC user” line on their resumes
A similar thing is going on with AI right now, but instead of using an honest term “confident AI users” people call themselves “AI experts”, and it kinda pisses me off
2
2
36
1,820
Jul 5
> Dell
2
16
2,518
Jul 2
Writing (good) English prose is orders of magnitude harder than writing code. Like not even close
22
8
168
11,512
Jul 2
She found french fries in the trash bag and played with them
37
546
Jun 30
The expected index of a sequence which has a probability p is 1/p. So you use -log(p) bits to encode it. Resulting in -p*log(p) average code length
1
1
4
600
Jun 30
Actually this unironically works if you enumerate the binary sequences using a pseudorandom number generator!
If the distribution of PRNG outputs matches the distribution of data sequences this gives an ~optimal entropy encoding
Pro tip: to save disk space, you can write a tool that sequentially generates all binary sequences up to a chosen length. Then, simply replace files on your disk with indices into that output stream.
The indices can get long. We address this by generating a list of indices and
3
20
3,088
Jun 30
Car-people are fascinating
You can take their stuff, send their families to camps, piss on their rugs, do whatever you want, and they will be complicit
But the moment gas prices go up the discontent goes through the roof
Jun 30
Russians spending their days searching for fuel complain of a sense of fear and panic now overwhelming society, as everything breaks down, with many saying that there will only be relief after the war.
3
19
1,350
Jun 30
I know very little about CUDA programming, so would love someone more competent to weigh in on the topic
2
286
Jun 30
Of course, it may be the case that GPU driver implements multiprocessing by time slicing, so every process gets full GPU. But afaik there are no guarantees anywhere that simply launching with num_ctas = num_gpus results in full residency
3
3
564
Jun 30
Basically the async megakernel runtime assumes that threadblocks are pinned to SMs and are always resident. If something happens and one threadblock gets suspended (for example because another process needs to use the same GPU), you can get stuck in an infinite spinlock
1
4
455
Jun 30
Btw nobody ever mentions this problem! From what I saw, the original HazyResearch codebase relies on prayer.
Maybe there are some CUDA scheduling contracts that I am not aware of, but this seems super dangerous
Jun 30
When writing megakernels, how do you avoid deadlocks when synchronising multiple CTAs? Are you using cudaLaunchCooperativeKernel, or just launching num_ctas = num_sms and relying on prayer and that nobody launches a second gpu process on the same machine?
2
37
5,742
Jun 29
While I was writing this thread a significant part of my twitter feed was discussing whether mathematics can be useful in life…
Jun 29
Applying the “second wonderful limit”, we get an asymptotic form for large n:
m(1 - exp(-n/m))
And by taylor expanding the binomial formula for n << m, we get
n - n(n - 1)/2m
Lets call n/m “saturation ratio”
1
13
3,379
Jun 29
What if in the attempt to distance from llm slopsters everyone would start writing in the manner of continental philosophers?
Jun 29
A good way to spot AI slop exists.
Read every sentence.
And think of the Vine Boom after each.
If it feels appropriate,
The writing is probably AI generated.
2
10
2,535
Jun 29
The big implication here is that "fastest inference" and "the most energy efficient inference" are not necessarily the same thing, because the speculative budget at which you acheive maximal interactivity is not necesserily optimal from the energy efficiency POV
2
276