General AI challenge, $5 million prizes in total

[FrankBuss]

Not strictly electronics related, but still technical and some programmers here might be interested in it: https://www.general-ai-challenge.org

They have a detailed description of the tasks and the github repository with test cases and agents in Python is up. 1st place for the first round is $15,000. You have time to write your solution until August 15, 2017.

I guess they don't mind if you do something more electronics related, like building a FPGA based neuronal computer, or using IBM's SyNAPSE chip. Maybe you don't win the challenge, but they might hire you, or give you an extra prize.

I'm not associated with the company who is organizing the challenge, I heard about it on the singularity podcast. And no worries, the company is called "Good AI", so no way that something like Skynet could be created

[FireFlower]

Yeah seems interesting enough with enough prize money on table, so going to pass the contest along.

[FrankBuss]

Yeah, good photoshopping But the original was a military helicopter, which you could argue is worse than a pretty actress: http://knowyourmeme.com/memes/select-all-squares

Yeah seems interesting enough with enough prize money on table, so going to pass the contest along.

Good luck. I read the specs now in detail and tried the micro-tasks with the human-interface of the Python code, and couldn't even solve the first task, without looking in the source code (but then I'm not good with such puzzles). But I think there are some more fundamental problems. I discussed it with the organizers on their blog, see the comments at the bottom here: http://blog.marekrosa.org/2017/02/first-round-of-general-ai-challenge_15.html

Now I don't think it is possible to win one of the quantitative prizes, but at least someone could have some fun trying to do it and might win a qualitative prize because of the jury wild-cards. The regular way to win a qualitative prize requires to solve all tasks, and be selected by the jury because of the quality of the AI.

[Mechatrommer]

adaptive learning AI? good luck, i havent download the detailed spec just reading the general description on the main page. have wonder why the 20 engineers in the organization in the business havent figured out that yet? if they were me i'd keep the 5 millions to myself and make my own code. you need to have a theoritical phd or either a fantasizing graduate for that, good luck.

[Brumby]

adaptive learning AI? good luck, i havent download the detailed spec just reading the general description on the main page. have wonder why the 20 engineers in the organization in the business havent figured out that yet? if they were me i'd keep the 5 millions to myself and make my own code. you need to have a theoritical phd or either a fantasizing graduate for that, good luck.

My guess is that formal thinking processes have reached their limits and that they are throwing the doors open for new blood and out of the box thinking.  I believe the hope is that someone is going to come up with some weird approach that is going to lead to new fields of development.


Sounds good, anyway.

[FrankBuss]

I've written a simple webserver so that everyone can try to solve the tasks with a webbrowser, instead of installing Python first:

http://www.frank-buss.de/ai/index.html

Do you manage to solve the second task? How many tasks can you solve? I couldn't even solve the first task without looking in the source code I hope there are some smarter AIs out there.

[PointyOintment]

I haven't even gotten a single successful guess on task 2, let alone the ten in a row needed to pass, after about ten minutes and over 2,223 keypresses (history depth you display * number of keys on my keyboard that I tried). I guess humans aren't as good as we think we are at learning how to learn

Edit: I did about 800 more guesses (10–15 times each key again, with shift held down this time) and still no +es.

Edit 2: I checked the spec [PDF] (spoilers!) and the task described in section A.2.2 doesn't seem to match what I was seeing. Also, it would be nice if the web interface would tell me I passed task 1 and am now working on task 2, so I don't have to count how many +es in a row I got and figure out if I passed each task and am now on a different one.

[FrankBuss]

I haven't even gotten a single successful guess on task 2

Well, this was tricky, my explanation was not quite right how and when the second task is chosen, but as an intelligent human you should have discovered this  I updated the webpage, now it explains it better. I can't show the current task, because in the challenge the AI program wouldn't see it either. But maybe I should add a checkbox to do this for us stupid humans

This is the second task:

https://github.com/general-ai-challenge/Round1/blob/master/src/tasks/challenge/round1/challenge_micro.py#L283

Translation: for all ASCII characters you have to answer with the same one random character, same as in the first task, but for the characters !":?.,; and space you have to answer the question character.

Looks like the first task can occur multiple times at the beginning before the second task is chosen. This task scheduler is used:

https://github.com/general-ai-challenge/Round1/blob/master/src/core/scheduler.py#L121

Edit 2: I checked the spec [PDF] (spoilers!) and the task described in section A.2.2 doesn't seem to match what I was seeing.

Looks like they have some problems with formatting in the PDF, the space character has not the same fixed width than the other characters. But it seems to be the second task as I described it.

But the tasks are just examples. They could use a complete different set for the non-public tasks when testing the AI programs.

[FrankBuss]

The organizer updated the rules document:

https://discourse.general-ai-challenge.org/t/important-round-1-specifications-and-rules-update-1/298

Now it is more clear, that you can win the qualitative prize without even writing a single line of code, if you submit a nice white paper with some ideas.

[PointyOintment]

Looks like DeepMind may have already won: http://www.wired.co.uk/article/deepmind-atari-learning-sequential-memory-ewc

[FrankBuss]

Looks like DeepMind may have already won: http://www.wired.co.uk/article/deepmind-atari-learning-sequential-memory-ewc

The article says "While it is able to learn from its previous experiences and retain the most useful information, it isn't able to perform as well as a neural network that completes just one game.". But looks like their neural network dedicated to one game couldn't even complete an old Atari 2600 game:

http://www.wired.co.uk/article/google-ai-montezuma-revenge

Granted, most humans perform worse, but come on, 100 million tries and it still can just explore 15 out of 24 rooms? Remember, to solve the AI challenge, the program has to solve all tasks, they will just measure how many steps it needs. No chance.

[PointyOintment]

Isn't this challenge a lot simpler than even old Atari games? And the rules update post makes clear that even if one program does better in an absolute sense, what they're going to reward is learning and improving, even if it's slower overall.

[FrankBuss]

Maybe you are right. The source code for the Atari 2600 playing AI:

https://github.com/kuz/DeepMind-Atari-Deep-Q-Learner

Anyone who wants to try it with this challenge?

[FrankBuss]

Nice, my webserver was featured in the weekly update:

https://www.general-ai-challenge.org/weekly-update-4

[FrankBuss]

Just an update, the first round is finished. As expected: https://www.general-ai-challenge.org/gradual-learning

"None of the submitted agents were able to complete the evaluation curriculum. The most successful agent got close to completing 25% of the tasks (see the chart). Therefore, the quantitative prize was not awarded in this round."

It is like with these intelligence tests with number sequences. There are just too many possible continuations, and these tests had even more degrees of freedom. This might be the reason that there were only 13 submissions at all. Maybe the next round is more interesting: https://www.general-ai-challenge.org/ai-race But unless I missed it, they didn't say anything about what tasks has to be solved for it?