Chat GPT AI

Deathwing said:

I'll admit some ignorance here. What about a neural network requires a process of that size?

Click to expand...

It's an equation with 314 billion parameters that need to be loaded. They do admit that the implementation isn't the most efficient possible.

"a machine with enough GPU memory is required to test the model with the example code. The implementation of the MoE layer in this repository is not efficient. The implementation was chosen to avoid the need for custom kernels to validate the correctness of the model."

Deathwing said:

IDK, I'm sure part/all of this is ignorance on my part. But even an equation that large seems like it could be partially paged out or cached.

Click to expand...

They obviously can be, since "the cloud" doesn't stock infinite 300 GB GPUs either and the problem is always split across discrete hardware. For some likely valid reason above my pay grade they couldn't or chose not to use those methods for this release.

Deathwing said:

The trend seems to be growing the amount of inputs. What do you do when the hardware simply doesn't exist for all this shit that needs to be jammed into GPU memory?

Click to expand...

The math for LLMs has been around since the 1960's. The reason the tech is in its current state today is precisely because the hardware now allows it. No one is building models that can't be run.

Wolfram has a good video that explores the space of a model. It is mostly empty. I'll see if I can find it.

Hmm I wonder if you could use a cubic run length encoding pass to squash empty regions?

derp, I posted it a month ago in the image thread. Generative AI Space and the Mental Imagery of Alien Minds

The islands normally seem to be roughly “spherical”, in the sense that they extend about the same nominal distance in every direction. But relative to the whole space, each island is absolutely tiny—something like perhaps a fraction 2–2000 ≈ 10–600 of the volume of the whole space. And between these islands there lie huge expanses of what we might call “interconcept space”.

Click to expand...

Kharzette said:

Generative AI Space and the Mental Imagery of Alien Minds

Click to expand...

That was a great piece. Steven Wolfram is the best kind of crazy and an excellent translator of complicated ideas to a lay audience.

NVidia introduced the new Blackwell GPU yesterday, which will give a massive boost to training AI models, making it faster and/or cheaper. They're also working on simulated worlds, so that AI's trained in them can build a world model. Yann LeCun mentioned this as a required step in the recent Lex Fridman podcast. Training them on language only can only go so far because language is an abstraction of the real world (a bit like how your brain cannot learn to ride a bike by reading books alone)

Very interesting stuff.



Full keynote address (2hrs)


Bullet points only

Lambourne said:

NVidia introduced the new Blackwell GPU yesterday, which will give a massive boost to training AI models, making it faster and/or cheaper. They're also working on simulated worlds, so that AI's trained in them can build a world model. Yann LeCun mentioned this as a required step in the recent Lex Fridman podcast. Training them on language only can only go so far because language is an abstraction of the real world (a bit like how your brain cannot learn to ride a bike by reading books alone)

Very interesting stuff.

View attachment 520511

Full keynote address (2hrs)


Bullet points only

Nvidia GTC 2024 — Jensen Huang keynote and all the day one news and releases as it happened: Blackwell, robots, Apple Vision Pro and more

Day one of Nvidia GTC 2024 has already seen some major releases

www.techradar.com

Click to expand...

Out of that entire video, the only thing that mattered to Reddit and other left wing MSM news is that Jensen said that young people should expect "rough times" out of context, which in context was talking about entry level jobs. At least this is what Ive gathered from other posters, someone can correct me cause Im not watching the whole thing, either.

Mist said:

If there's no entry-level jobs, there's no way to get job skills for the non-entry level jobs. No amount of education is a substitute for real operational work experience.

Every field has an entry-level, even advanced professions. Break that pipeline and shit gets fucked fast. We already have a problem where companies want 10 years' experience in 4-year-old technologies to get an entry-level position. A generation of kids who can't get on the ladder is a disaster. We already got a brief glimpse of that during the last financial crisis.

Click to expand...

I realize that entry level jobs has a wide girth of opportunities. But I'm not envisioning a single job I had that would be classified as "entry level" that spurned me on to a career in a master class field. Even examining others, I'm having trouble. What jobs would you say will be replaced that are integral in the chain linked to careers? I cant imagine putting Pizza Hut at 16 down on my resume helped me in any capacity.

Daidraco said:

Out of that entire video, the only thing that mattered to Reddit and other left wing MSM news is that Jensen said that young people should expect "rough times" out of context, which in context was talking about entry level jobs. At least this is what Ive gathered from other posters, someone can correct me cause Im not watching the whole thing, either.

Click to expand...

I watched the whole thing and I don't think he talked about jobs anywhere.

Mist said:

What part of "every field has an entry level" did you not understand? If you automate all the low-level engineering or programming jobs, where do you get your future principal engineers and software architects from? If you automate all the copywriting jobs, where do you get senior product managers from?

Click to expand...

You are such an insufferable fucking cunt 99% of the time. I asked you, without sarcasm, which jobs you're referring to because "I" am not thinking of the entry level positions you speak of, and thats how you respond. Yet you fucking complain about how people "act" here.

I dont think for a fucking second that without basic understanding of the concepts, that a person will be able to use the AI to build all but the most basic fucking programs using AI. No matter how advanced the AI gets. Will AI put a dent in the workload and available positions for these "so called" entry level positions? Most definitely. But, as respectfully as I can be, you fucking wanna-be tranny, I did not consider fucking "programming" to be on par with fucking Flipping Burgers, as an entry level position. It's also asinine to think that if these jobs did disappear, that no other avenue would appear when companies are looking to hire the "Senior product managers" .. and if there is no need for them, then I guess thats the fucking end of that, too! As if specific jobs or entire industries have not withered and died in the past. Not that I believe for a fucking second, unlike you apparently do, that that will fucking happen.

As Booze would say,

Lambourne said:

I watched the whole thing and I don't think he talked about jobs anywhere.

Click to expand...

Master cunt knew exactly what I was talking about. I had to look in my history and go back to the thread, but its a click bait title and its just talking about programming jobs.

AI is coming to replace us, bros.