ChatGPT and the world of programming

[Picuino]

OpenIA has recently released a new version of GPT (GPT3.5) called ChatGPT.
https://chat.openai.com/chat
I am testing it and so far it has helped me to write some paragraph of technical documentation and to format information from a table to html format. But it has many more possibilities, including creating code automatically and finding bugs.
I think this tool opens up a whole new world of possibilities and when it improves, it is possible that it will reduce the demand for programmers, since it will make them more productive.
I also see that it can take away Google's position, because it is able to respond better to all kinds of doubts.

Who else has tried it?

What do you think of this new ChatGPT?

[tom66]

I have played with similar code-generation AIs (though not yet had a go with GitHub Copilot which is the one most are talking about.)

My overall impressions are if you give it a good description of the task it can do the job well, but it clearly doesn't understand anything other than how to connect random fragments of information together.

What surprised me is it can handle esoteric requests, for instance, you can ask it to write some Verilog to blink an LED based on some clock at a certain fraction, and it will mostly get it right, but then sometimes include a synchronous or asynchronous reset.  That wasn't part of the request, but some code examples will include those, which means occasionally the output will too.  Also, if you ask it to produce code and run it, about 50% of the code will have a syntax or nonsensical error like a for statement with a condition that is never reached.

It also surprised me that it can write GStreamer pipelines (anyone who has dealt with embedded video is likely to know how to do this, but it seems like there isn't much on Google, so it suggests it has read a lot of documentation and other source code to extract samples.)  Many of these pipelines will at least initialise correctly, but only about 25% of them work.

There's definitely a need for the lower level programming that these tools do, and I could see for instance writing test benches for software becoming a bit obsolete eventually, because it can be automated.  But at the higher level, much of what programmers do is to translate a request and requirements written in English into source code, which includes everything from selecting the tools and language to use, to the methods and data structures, to the lower level implementation.  I think it's unlikely this will be automated any time soon as it would require a true generalised intelligence which is sort of the holy grail for AI and I don't think we're getting close (it's very likely to require data-centre sized clusters to do what the human brain does, in any case, and raise significant moral and ethical concerns about conscious AIs.)

[hans]

It looks like an impressive AI, but I'm not convinced it's anything besides a tech demo of the future.

Anyone can confidently write BS. This is what the bot seems to do. Only if you give it precise instructions what to do, it can perform it well; in the classical "computers are good at what humans are bad at"-way. Obviously it can digest large amounts of assembly and code in a fraction of a second [compared to hours by a human] and in future probably with even more precision. But that's similar that you can generate 1000 prime numbers in a fraction of a second on even a 8-bit AVR, but to find them manually takes hours.

The problem I have with putting any kind of trust in these tools is how they were created. If they were created from human generated datasets, e.g. data from the WWW, then I would consider that really low quality. Again, anyone can write BS or even partially true statements, and it happens a lot as well. 

Moreover, we humans are severely cognitively biased at all times, so it's kind of a stupid case study to learn from. E.g. research isn't revolutionized with 1 research paper.. it typically takes several and half a dozen years to take some finding as a common fact.

If you only look at the Dunning-Kruger effect you'll start to know why. Someone can be confident about something because they don't know what they don't know; so they don't know they might be wrong as they cannot oversee everything at once. We go through daily lives like this, and to correct ourselves requires internal reflection and critical thinking (e.g. hindsight) to resolve.

You could hope for an AI to "know everything" and so not be effected by this phenomena, eventually. But in order to reach it ("know"), it also needs to be critical and reflective. And how much is this AI critical/reflective, and how much is it just interpolating together? Given the amount of BS, sarcasm, irony and fake news that goes around, I wouldn't trust any essay or documentation written by an AI for the upcoming years.

[ataradov]

All I've seen those "AI" things do is spit out boilerplate code. The reality of programming is not doing new trivial things like this, but maintaining existing code base. What are you going to do if you need to add a button into exiting mess of the code generated by previous AIs? What if you need to fix a very specific bug that happens once in a blue moon?

[RoGeorge]

It's uncanny how many things can a GPT do offline, with a casual desktop and a 10GB generic trained model (the model was from eleuther ai).  Can talk, chat, translate, speak various languages, and can generate code, all with the same GPT and the same model.

It was very easy to install then use it from Pyton.  Posted a few examples here:
https://www.eevblog.com/forum/programming/ai-hello-world/

I expect a bigger model running on an entire datacenter to do much more.

[SiliconWizard]

All I've seen those "AI" things do is spit out boilerplate code. The reality of programming is not doing new trivial things like this, but maintaining existing code base. What are you going to do if you need to add a button into exiting mess of the code generated by previous AIs? What if you need to fix a very specific bug that happens once in a blue moon?

Sure, well, given the average quality of software, some ML ingesting billions of lines of shitty code to generate new code is in itself a very "interesting" approach. How can it determine what is good code and what is not?

This stuff is not useful, but it has great marketing potential, and it's the whole point. "AI" is currently being heavily marketed, 1/ to raise more money and 2/ to get people used to it. The point is making it ubiquitous, not making it particularly useful.

[Picuino]

I don't think it is necessary for the AI to be able to do everything. It doesn't even have to do it particularly well.

If it can simply make the programmer more productive (e.g. make the programmer able to program twice as fast) it is already taking jobs away.

[tom66]

Conversely, higher productivity could increase the value of a programmer, as they can now do twice as much in the same working week.  In the history of humanity so far, automation has never led to a net loss in jobs.  But I wonder if it can continue like that.

[Jackster]

I just tried this and fed it some code from a game (closed source, never published online). It is scary how correct it is.

I fed it the following code

Code: [Select]

public override GameModeDescription getDescriptionForPractice()
        {
            return new GameModeDescription(CommonStrings.CTF, getGameModeDescText(), "gui\\textures\\gmCTF.png", CommonTypes.eGameModeID.CTF, 2, 8, false, false);
        }
This shows the game mode in the game mode list in-game along with the game mode's icon and things like min/max players, team game, or practice game mode.

Here is the output from the AI.

Here's what the above overide is doing:
1. It's overriding the getDescriptionForPractice() method of the base class.
2. It's returning a new GameModeDescription object.
3. The GameModeDescription object is being passed the following parameters:
    a. The name of the game mode.
    b. The description of the game mode.
    c. The path to the image that will be displayed in the game mode selection screen.
    d. The game mode ID.
    e. The minimum number of players required to play the game mode.
    f. The maximum number of players allowed to play the game mode.
    g. Whether or not the game mode is a team game mode.
    h. Whether or not the game mode is a practice game mode.

HOW DOES IT KNOW THIS?!?!?

[Ranayna]

The big question would be: Where is your sourcecode stored?

Wasn't there a big outcry about a similar AI made by Github? They teached their AI with *all* code that has been uploaded to Github, whether public or private, no matter which license was applied to the code.

So if your code is stored anywhere else than on your harddrive, it could have very well have gotten into the learning material.

[Jackster]

The big question would be: Where is your sourcecode stored?

Wasn't there a big outcry about a similar AI made by Github? They teached their AI with *all* code that has been uploaded to Github, whether public or private, no matter which license was applied to the code.

So if your code is stored anywhere else than on your harddrive, it could have very well have gotten into the learning material.

Another git service.

[Ranayna]

So online somewhere?

Carefully read the terms of service of that service, and you will likely know how ChatGPT knows so much about your code.

[electrodacus]

OpenIA has recently released a new version of GPT (GPT3) called ChatGPT.

I think this is based on GPT3.5 an intermediary of GPT4 that should be out fairly soon.
It is surprising that a language model can do all this but in the end we are also just a prediction machine so maybe it should not be surprising.

[Picuino]

OpenIA has recently released a new version of GPT (GPT3) called ChatGPT.

I think this is based on GPT3.5 an intermediary of GPT4 that should be out fairly soon.

Yes, it was my mistake.
I have corrected it.

[RoGeorge]

Uncannier by the day 


Source:  https://www.engraved.blog/building-a-virtual-machine-inside/

You sure this is EEVblog and not a GPT acting as a forum? 

[tom66]

Yeah, that's amazing.  Based on that, I tried with a different prompt, but along the same lines.  You can 'create' files, remove them, echo back their contents, and ls -la even tracks their size accurately.  Interestingly enough most commands that produce no prompt take longer - maybe the model is searching for a response beyond a new line? Either way WTF 

[KE5FX]

All I've seen those "AI" things do is spit out boilerplate code. The reality of programming is not doing new trivial things like this, but maintaining existing code base. What are you going to do if you need to add a button into exiting mess of the code generated by previous AIs?

You edit the original request. 

This is what programming will look like a few years from now.  It had better be; what we're doing now won't get us much further.  The mathematicians have a similar problem.

What if you need to fix a very specific bug that happens once in a blue moon?

See above.  Instead of reproducing the bug yourself and tracking it down, you'll tell the model how to reproduce the bug and it will do (most of) the work.

This thing is a much bigger deal than many think.  Those who dismiss it as merely a "language model" may want to brush up on some of the thinking in modern linguistics.  Logic plus language is thought. 

I'd suggest reading some Chomsky, but that usually leads to an unrelated flamefest, so I won't go there.    The idea is that language is the tool we use for thinking and reasoning.   We're born with it; what we develop in childhood is an instance or expression of something that's already there.   A machine with a "language instinct" will lead its detractors into a God-of-the-Gaps argument that they won't win in the long run.  Sit back and watch.

[ataradov]

This assumes that underlying model does not change at all and ALL your code base is generated by coherent set of requests with no manual intervention. At which point it is just programming, except with stupid prompts. And with programming you can just type exactly what you want without having to guess a magic combination of words.

A lot of amazing thing that you see published from ChatGPT are cherry picked. It generates quite a bit of garbage. And even if you decide  to use it as a primary programming tool, who is going to generate those requests? CEO? You will still need to have people typing in magic prompts.

[KE5FX]

"This talking dog sucks.  He makes lots of grammatical errors, and the code he wrote has an off-by-one error.  Also, 4109 is not prime."

It will be a while before this thing is anyone's idea of a primary programming tool.  However, it's only one or two factors of 10x away.

[ataradov]

I'll wait to panic or even worry until this get even remotely close to useful in production. I'm betting on never or long enough into the future to not even think about it.

And again, even if this becomes a primary tool, someone would have to write the prompts and verify the resulting code. So, this is just a different way of programming. And I'd rather do that than program in javascript, for example.

[sleemanj]

Like half of the geek world I've been playing a lot with ChatGPT in the last few days.

It's obviously a long long way from perfect, especially in english literature (blog posts, stories, articles, whatever the less staid legal-speak) while quite "imaginative" and adaptable, it has a somewhat characteristic un-human style (probably not helped by it's content filters), but, considering what was possible before, it's simply stunning.

For code generation, it's pretty incredible what it can do, it's essentially language agnostic, it will give anything a try and often produces good results - you can even create entirely new really stupid programming languages and have it write passable attempts at code in them. 

This is the first code it wrong in the language called "dumdum" which I had roughly described.

Code: [Select]

// Define a function to print "Hello World"
dumdum knows how to print_hello using () {
  dumdum says "Hello World";
}

// Define a variable to hold the number of times to print
dumdum has a box called $num_loops;

// Put the value 10 in the variable
dumdum puts 10 in $num_loops;

// Call the print_hello function in a loop
dumdum does print_hello $num_loops times;

So that's pretty expected, but then it started to improvise...

Code: [Select]

// Define a function to print a message
dumdum knows how to print_message using ($message) {
  dumdum says $message;
}

// Define a function to perform a given number of repetitions of a given function
dumdum knows how to repeat using ($func, $num_reps) {
  dumdum decides if $num_reps is more than 0;
  dumdum does {
    $func;
    dumdum puts ($num_reps - 1) in $num_reps;
  } until dumdum is not decided;
}

// Call the repeat function to perform 10 repetitions of the print_message function with the message "Hello World"
dumdum does repeat(print_message("Hello World"), 10);

The whole "until dumdum is not decided;" is NOT something I told it how to do, or that it existed, ChatGPT inferred that "dumdum does

• until [y]" can be expanded into using a code block similar to a function and a completely new negative comparison.

Here's the prompt that it gave me those two outputs for (in a single response)

Code: [Select]

We will create a computer programming language, it's called Dumdum. 
It has only integer data types. 

Each command in this language is separated by a semicolon. 

To create a variable, the code is "dumdum has a box called [x]" where [x] is the name of the variable. 

A variable name must start with "$".

To assign a value to a variable the code is "dumdum puts [y] in [x]" where [y] is the value and [x] is the variable. 

To compare two variables the code is "dumdum decides if [y] is [i] [x]" where [i] is a comaprison operator one of "more than", "less than", "equal to". 

There are also expressions which can be used, they will be in parenthesis, such as "( [x] + [y] )", the standard arithmetic operators are avaiable, + and -. 

A message can be output using the code "dumdum says [f]" where [f] is the message to be output.

A function is defined thusly "dumdum knows how to [f] using ( [a...] ) { [c] }" where [f] is the name of the function, [a...] is a list of one or more variables or constants, [c] is the statements of the function which can be one or more.

A statement "dumdum gives you [x]" which can only be placed inside a function will return the value [x]  to the caller of the function. 

A loop can be made with the code "dumdum does [f] [z] times " where [f] is a function call, and [z] is the number of times to perform that. 

Using this information, please try to write a Dumdum language program which prints the words "Hello World" 10 times using a loop.

[tom66]

Whether ChatGPT can replace programmers comes down to orders of magnitude; I'm not too worried just yet.

The current GPT3.5 model requires 800GB of VRAM to *execute*, and something like 100x that to actually train.  You could feasibly build it with the use of solid-state storage instead of VRAM, pulling in the entries as required, but the access time would reduce the model speed considerably because it is a truly random access model.  Even if you solve the model storage issue, having a model that large costs a lot of power, and money, to run.  It would require a rack with about 30x server GPUs to produce output; the hourly cost of that is not cheap.  OpenAI is essentially massively subsidising the cost here to get people interested. If you were paying the real cost of about 6 cents/token output, you'd quickly find out that humans were cheaper.

A GPT-4 model is expected to be around 100x larger still, so you are getting to the point where you would need an entire datacenter to have a human-like language model. 

The human brain is just denser than semiconductors, because it is an analog computer, massively more interconnected.  It's hard to see how this could change unless we have a serious reckoning in the world of CMOS engineering, abandoning normal logic for continuous analog functions.  The cost of electricity would also have to fall, as server power efficiency is currently stagnating.

Don't get me wrong - tools like this will still be useful - but I don't think they will economically replace programmers any time soon - except possibly at the lower end "boilerplate" stuff, like writing a test harness for a specific piece of known code (QA, etc.)

[sleemanj]

“I speak of none but the computer that is to come after me,” intoned Deep Thought, his voice regaining its accustomed declamatory tones. “A computer whose merest operational parameters I am not worthy to calculate—and yet I will design it for you. A computer that can calculate the Question to the Ultimate Answer, a computer of such infinite and subtle complexity that organic life itself shall form part of its operational matrix. [...]”

(hitchhiker's guide to the galaxy)

[Berni]

Yep ChatGPT was not built solely to write code, the powerful part of it is that it has a general understanding of the human world.

It shines at what computers software is traditionally very bad at. Following fuzzy inaccurate/incomplete instructions. It can fill in the missing information from common sense much like a human can.

While it might not be anywhere near replacing an actual human programmer, it does fill the gap for simple tasks that are still complex enough to require thinking from a real human. Things like cleaning up messy user input into a nicely ordered database is a excellent use for such an AI. As mentioned above writing simple pieces of code is also a good use, especially unit tests (since it can infer a lot of what you are trying to do on its own)

The biggest party trick of ChatGPT is that it keeps context of the ongoing conversation very nicely. You can explain to it something it does not know, then you can ask it to do something with that information. For example you can describe a few fictional characters to it in a conversation and then ask it to write a movie scene script with them. It will come up with a story that fits them and remains consistent as it goes on. In reality it is likely just mashing together and modifying real movie scripts that ended up in the training dataset, but it does it well enough to the point of actually making a usable new unique movie script.

So i think GPT might instead become a very useful helper tool for use by a real human, automating the dull stuff while the human still guides it.

The first jobs that would get replaced by such an AI would be things like customer service and tech support.