Local AI and hardware to run it

[dietert1]

When i read these reports of using AI for software development i am asking myself how those involved think they own their results. Is this a problem? How is it solved? Running models locally may be an attempt to solve that problem.
I mean the day before we had this doomsday kind of warning where the superintelligent AI wants to enslave humans as its human intelligence agents. And appearently those even pay for being engaged. Fantastic.

Regards, Dieter

[brucehoult]

So, don't assume that it "can't test it."  Unless you are working on something seriously weird and/or dangerous, if people can test it, so can the clanker.

IFF the human can test it without physically plugging and unplugging cables, pressing DFU buttons etc.

[tom66]

When i read these reports of using AI for software development i am asking myself how those involved think they own their results. Is this a problem? How is it solved? Running models locally may be an attempt to solve that problem.
I mean the day before we had this doomsday kind of warning where the superintelligent AI wants to enslave humans as its human intelligence agents. And appearently those even pay for being engaged. Fantastic.

Regards, Dieter

Code created by an AI model is, by definition, not copyrightable, because it was not created by a human and copyright only applies to human work.

But, if a human reviews that work, alters it, and iterates the agent or chatbot's output, then it is copyrightable.

[tom66]

One "benefit" of the current situation is that HW keeps it value better than ever. In 1997 or 2010 you would have been a fool to buy a "too powerful" set of computer hardware, you would rather buy "just enough" for the job because if you find out you need more RAM / more CPU / more disk / whatever after just 6 months, you get the upgrade for much cheaper. Any investment lost half of its value in a year.

Now, you can buy the Mac, or the GPUs, or whatever you want if you have the cash. They'll keep their value like you were buying gold, good chances the value could even increase. Feels ridiculous but that what it is. So why not just buy the Mac, if it turns out to be a wrong move, just sell it for ~ the same money, or in a good case, for more.

This is going to be my argument to my boss when I ask him to spend £25k on Apple hardware; we shall see!

[Siwastaja]

When i read these reports of using AI for software development i am asking myself how those involved think they own their results. Is this a problem? How is it solved? Running models locally may be an attempt to solve that problem.

I consider that I own the work I do with AI, just like I consider I own the binary code made by compiler from source code written by me. What "ownership" means is purposely vague - I don't care. I also don't care if others disagree. I'm used to the fact that significant % of human beings are stupid idiots that can be ignored.

And the Chinese will copy it anyway, and you can't stop it.

The question is theoretically interesting, a perfect coffee room discussion. Whether the model was ran locally or not doesn't play any role in it whatsoever. If politicians or lawyers decide that AI work is uncopyrightable, which is an interesting disagreement compared to how code generated by compilers is, and if they find a way to somehow define a clear line between AI "assisted" coding (complexity of the prompt, amount of hand-holding, amount of reviewing and modifications) or "AI designed" (less of those factors) code, then surely the exact same conditions will apply to local computing. Why would the ownership status of the computer (hired / self-owned) affect anything?

If anything, there will be a weak correlation because local models are less capable and as such more human work will be involved, but all of this is nearly impossible to prove.

Therefore, keep doing good work, keep advancing the society, and ignore envious idiots.

[Siwastaja]

Code created by an AI model is, by definition, not copyrightable, because it was not created by a human and copyright only applies to human work.

Why binary code created by a compiler is copyrightable, then?

[ejeffrey]

Code created by an AI model is, by definition, not copyrightable, because it was not created by a human and copyright only applies to human work.

Why binary code created by a compiler is copyrightable, then?

tl; dr: it is not.

I don't know about the rest of the world but the US copyright office current statement is that prompting an AI model in a similar fashion as to how you would prompt a human artist or writer does not confer copyright.  If it's a human creator, the copyright legally falls to them unless there is a contract for ownership transfer.  Whether the way people are using AI coding agents falls into that camp, or whether it is using a machine assistant would be a case by case evaluation.  Mostly the existing case law considers AI image generators like midjourney.  In that case, it's very clear in the current law that if you ask it for "a photorealistic image of a unicorn jumping over the moon with the auroras in the background" the resulting image is not copyrightable in the US.  If you did significant creative post-processing, the result could still be copyrighted.

Compiled code doesn't really have or need a separate copyright from the input source code.  Which is mostly good.  The original creation is still copyrighted.  The ability to make mechanical transformations, whether compilation or simple copying is all covered by the original copyright.  As far as the copyright office is concerned, that's basically like scanning a printed book into a PDF.

Whether a piece of work is copyrightable completely different than whether it can infringe a copyright.  An AI output can 100% be copyright infringement, even if it is not itself copyrightable, for instance if the AI image generator spits out an image that is barely changed from it's training data.

[tom66]

Code created by an AI model is, by definition, not copyrightable, because it was not created by a human and copyright only applies to human work.

Why binary code created by a compiler is copyrightable, then?

The source code that made the program is copyrightable (providing it was generated at least in part by some human effort), and the process of transforming it through a compiler is seen in law as similar to how a printing press produces a book; the author still retains their copyright even if the book was printed by a machine.  A good case is Apple v. Franklin (1983) which established this fairly firmly in law.

I think what this means is that whilst an AI model is copyrightable (since it was created by humans) the output of a model is not copyrightable because the process of running the model is sufficiently transformative, i.e. it is not simply taking one form of code and converting it into another.  Therefore, it's counted as being in this strange space between not being generated by a human and not being copyrightable.

It's a bit of a novel area though so watch this space.

[Swake]

I think what this means is that whilst an AI model is copyrightable (since it was created by humans)

Mmmm, that is an interesting view going to generate interesting discussions.

In my opinion an LLM model must also be considered as a 'computed' artifact as it is created very similarly to how an executable is compiled. There is no human involved in the training process. Both are based on copyrighted 'code' and/or 'content'.

If the argument is followed that 'a compiled program is not copyrightable' then an LLM is not either.

[dietert1]

Independent of the legal situation what i learned in the past is software development needs to be on an offline machine.
Otherwise you will waste your time and money with safety measures and/or legal conflicts. Or your work may be stolen without any notice.
Some AI experts say the next years until the end of the decade will be very interesting but after that humans will suffer in every way you can imagine.

Regards, Dieter

[tom66]

I think what this means is that whilst an AI model is copyrightable (since it was created by humans)

Mmmm, that is an interesting view going to generate interesting discussions.

In my opinion an LLM model must also be considered as a 'computed' artifact as it is created very similarly to how an executable is compiled. There is no human involved in the training process. Both are based on copyrighted 'code' and/or 'content'.

If the argument is followed that 'a compiled program is not copyrightable' then an LLM is not either.

Yes, but the model does not train itself.  There are automated parts of the training process but also substantial human effort through reinforcement learning.  The model architecture also needs to be designed; models are not simply a pile of weights.  There are layers, attention mechanisms, KV cache entry/exit points, different tokenisation strategies etc.  So a model is copyrightable because it required noticeable human effort and creativity to design and create.

I think a good example would be quantisation of a model, which is an automated process.  If a company released an fp16 model as public domain or a lax licence like MIT, then someone who quantises a model, a computationally difficult process for large models but fundamentally a simple process, they could not copyright the output model.  Since all they did is run a "simple" command to convert the model from fp16 to int8, there was no creativity involved, and almost no effort. (Note that MIT does not prohibit private or commercial redistribution of modified sources, but it would be different if GPL was involved... that said, GPL would likely also require sources to the model to be released, too.)

[iMo]

.. We're currently paying about £100/m for AI credits per engineer and have five engineers using that..

..why to hurry?? ..another option could be to stay with those payments (which may go down too) till the local HW prices will go down significantly (I would expect end of 2027).. it may also mitigate the risk of buying the low performance local HW today..

PS: your developers will not be happier with those small local models either, on any local platform, imho..

[tom66]

.. We're currently paying about £100/m for AI credits per engineer and have five engineers using that..

..why to hurry?? ..another option could be to stay with those payments (which may go down too) till the local HW prices will go down significantly (I would expect end of 2027).. it may also mitigate the risk of buying the low performance local HW today..

PS: your developers will not be happier with those small local models either, on any local platform, imho..

You think hardware will get -cheaper-?  I think the exact opposite.  TSMC is booking up to 2028 right now.

[Siwastaja]

You think hardware will get -cheaper-?  I think the exact opposite.  TSMC is booking up to 2028 right now.

I believe it will - eventually. 2027 is of course optimistic. I wouldn't be too pessimistic, either, no one can predict the market dynamics of 2028. While AI is a significant change, significant supply vs. demand balance disruptions have happened before, and they always settle in 2-3 years.

But, ultimately, I don't think it's the local AI's time yet - capability gap is real, and the fast development keeps that gap open, hardware is temporarily (whether it's 3 or 4 or 5 years, it eventually is temporary anyway) very expensive. Local AI is a desirable direction, sure, but for now we are "stuck" with choosing between modern-day AI, a half-a-year behind Chinese pirated copy of it ran on a $200k local hardware (or in questionable datacenters - the worst option in my mind), or a smaller, simpler model limited to certain types of short, less autonomous mechanical tasks ran on a $5-10k local hardware.

Things could be different in 2 years. Maybe the race to AGI and singularity is a bubble which bursts. Maybe there is a fundamental capability limit to what LLMs can do and maybe the state-of-the-art is close to that after all. Maybe the whole market crashes and "frontier" research is just efficiency research. Maybe Fable 5.5 or GPT 6.5 is similar in capabilities but double the speed and half the RAM, and maybe Chinese copy those too, and maybe you can run that on a then-$50k machine, who knows. Maybe not that many are interested, maybe RAM is produced in excess again and sold dirt cheap. Not likely, but entirely possible.

[iMo]

TSMC is not producing majority of ddr5/6 rams and ssd/flash mems afaik.
The ddr5 and flash mems create the current price bubble..

There is indication the ddr5 mems from china's production get certified by several mobo makers these days.. Also they produce quality flash mems for ssd disks, and ramping up (today only perhaps 6-7% market share)..

PS: since early 80ties I've been messing with PCs I saw at least 5 such "memory crises" where the prices ramped up by several hundred percent, and then dropped way below the initial price within a year-two..

..You think hardware will get -cheaper-?  I think the exact opposite.  TSMC is booking up to 2028 right now.

[Siwastaja]

PS: since early 80ties I've been messing with PCs I saw at least 5 such "memory crises" where the prices ramped up by several hundred percent, and then dropped way below the initial price within a year-two..

I remember the earthquake-fab-destruction "crisis" in the late 1990's when 64MB was all the rage and 3D gaming was a relatively new thing. More recent examples are Covid-triggered chip shortage (all semiconductors, surprisingly RAM was among the least affected I guess), and before that, the MLCC crisis. Both recent cases are rather underestimating the demand and highly optimizing the manufacturing not to have any slack but operate as close as 100% duty as possible - just like the current situation.

The way out is either the increase in RAM production, or the demand bubble bursting. The former is slower, since investments are big and investors carefully want to see that the demand is real and permanent before increasing supply. Demand bubble can burst more quickly.

[iMo]

[Zondar]

I wish for a vigorous, on-going, technically-oriented discussion on this topic in this forum. Trying to wade through the fetid garbage-heap that is Reddit is a nightmare.

(I also wish I had bought an RTX Pro 6000 or two a few months ago when they were $8k instead of $15k+, but that's water under the bridge. )