Author Topic: General AI challenge, $5 million prizes in total  (Read 6557 times)

0 Members and 1 Guest are viewing this topic.

Offline FrankBussTopic starter

  • Supporter
  • ****
  • Posts: 2365
  • Country: de
    • Frank Buss
General AI challenge, $5 million prizes in total
« on: March 06, 2017, 12:28:41 pm »
Not strictly electronics related, but still technical and some programmers here might be interested in it: https://www.general-ai-challenge.org

They have a detailed description of the tasks and the github repository with test cases and agents in Python is up. 1st place for the first round is $15,000. You have time to write your solution until August 15, 2017.

I guess they don't mind if you do something more electronics related, like building a FPGA based neuronal computer, or using IBM's SyNAPSE chip. Maybe you don't win the challenge, but they might hire you, or give you an extra prize.

I'm not associated with the company who is organizing the challenge, I heard about it on the singularity podcast. And no worries, the company is called "Good AI", so no way that something like Skynet could be created ::)
« Last Edit: March 06, 2017, 12:33:16 pm by FrankBuss »
So Long, and Thanks for All the Fish
Electronics, hiking, retro-computing, electronic music etc.: https://www.youtube.com/c/FrankBussProgrammer
 

Offline FireFlower

  • Regular Contributor
  • *
  • Posts: 130
  • Country: fi
  • Wish had a cabin in Lappland
Re: General AI challenge, $5 million prizes in total
« Reply #1 on: March 07, 2017, 04:35:37 pm »


Yeah seems interesting enough with enough prize money on table, so going to pass the contest along.
 

Offline FrankBussTopic starter

  • Supporter
  • ****
  • Posts: 2365
  • Country: de
    • Frank Buss
Re: General AI challenge, $5 million prizes in total
« Reply #2 on: March 07, 2017, 08:27:54 pm »


Yeah, good photoshopping :) But the original was a military helicopter, which you could argue is worse than a pretty actress: http://knowyourmeme.com/memes/select-all-squares

Yeah seems interesting enough with enough prize money on table, so going to pass the contest along.

Good luck. I read the specs now in detail and tried the micro-tasks with the human-interface of the Python code, and couldn't even solve the first task, without looking in the source code (but then I'm not good with such puzzles). But I think there are some more fundamental problems. I discussed it with the organizers on their blog, see the comments at the bottom here: http://blog.marekrosa.org/2017/02/first-round-of-general-ai-challenge_15.html

Now I don't think it is possible to win one of the quantitative prizes, but at least someone could have some fun trying to do it and might win a qualitative prize because of the jury wild-cards. The regular way to win a qualitative prize requires to solve all tasks, and be selected by the jury because of the quality of the AI.
So Long, and Thanks for All the Fish
Electronics, hiking, retro-computing, electronic music etc.: https://www.youtube.com/c/FrankBussProgrammer
 

Online Mechatrommer

  • Super Contributor
  • ***
  • Posts: 11537
  • Country: my
  • reassessing directives...
Re: General AI challenge, $5 million prizes in total
« Reply #3 on: March 08, 2017, 02:18:27 am »
adaptive learning AI? good luck, i havent download the detailed spec just reading the general description on the main page. have wonder why the 20 engineers in the organization in the business havent figured out that yet? if they were me i'd keep the 5 millions to myself and make my own code. you need to have a theoritical phd or either a fantasizing graduate for that, good luck.
Nature: Evolution and the Illusion of Randomness (Stephen L. Talbott): Its now indisputable that... organisms “expertise” contextualizes its genome, and its nonsense to say that these powers are under the control of the genome being contextualized - Barbara McClintock
 

Online Brumby

  • Supporter
  • ****
  • Posts: 12288
  • Country: au
Re: General AI challenge, $5 million prizes in total
« Reply #4 on: March 08, 2017, 02:47:55 am »
adaptive learning AI? good luck, i havent download the detailed spec just reading the general description on the main page. have wonder why the 20 engineers in the organization in the business havent figured out that yet? if they were me i'd keep the 5 millions to myself and make my own code. you need to have a theoritical phd or either a fantasizing graduate for that, good luck.

My guess is that formal thinking processes have reached their limits and that they are throwing the doors open for new blood and out of the box thinking.  I believe the hope is that someone is going to come up with some weird approach that is going to lead to new fields of development.


Sounds good, anyway.
 

Offline FrankBussTopic starter

  • Supporter
  • ****
  • Posts: 2365
  • Country: de
    • Frank Buss
Re: General AI challenge, $5 million prizes in total
« Reply #5 on: March 10, 2017, 03:48:58 am »
I've written a simple webserver so that everyone can try to solve the tasks with a webbrowser, instead of installing Python first:

http://www.frank-buss.de/ai/index.html

Do you manage to solve the second task? How many tasks can you solve? I couldn't even solve the first task without looking in the source code ::) I hope there are some smarter AIs out there.
So Long, and Thanks for All the Fish
Electronics, hiking, retro-computing, electronic music etc.: https://www.youtube.com/c/FrankBussProgrammer
 

Offline PointyOintment

  • Frequent Contributor
  • **
  • Posts: 327
  • Country: ca
  • ↑ I scanned my face
Re: General AI challenge, $5 million prizes in total
« Reply #6 on: March 13, 2017, 08:29:59 am »
I haven't even gotten a single successful guess on task 2, let alone the ten in a row needed to pass, after about ten minutes and over 2,223 keypresses (history depth you display * number of keys on my keyboard that I tried). I guess humans aren't as good as we think we are at learning how to learn :P

Edit: I did about 800 more guesses (10–15 times each key again, with shift held down this time) and still no +es.

Edit 2: I checked the spec [PDF] (spoilers!) and the task described in section A.2.2 doesn't seem to match what I was seeing. Also, it would be nice if the web interface would tell me I passed task 1 and am now working on task 2, so I don't have to count how many +es in a row I got and figure out if I passed each task and am now on a different one.
« Last Edit: March 13, 2017, 08:46:43 am by PointyOintment »
I refuse to use AD's LTspice or any other "free" software whose license agreement prohibits benchmarking it (which implies it's really bad) or publicly disclosing the existence of the agreement. Fortunately, I haven't agreed to that one, and those terms are public already.
 

Offline FrankBussTopic starter

  • Supporter
  • ****
  • Posts: 2365
  • Country: de
    • Frank Buss
Re: General AI challenge, $5 million prizes in total
« Reply #7 on: March 13, 2017, 12:07:52 pm »
I haven't even gotten a single successful guess on task 2

Well, this was tricky, my explanation was not quite right how and when the second task is chosen, but as an intelligent human you should have discovered this  ;) I updated the webpage, now it explains it better. I can't show the current task, because in the challenge the AI program wouldn't see it either. But maybe I should add a checkbox to do this for us stupid humans :-DD

This is the second task:

https://github.com/general-ai-challenge/Round1/blob/master/src/tasks/challenge/round1/challenge_micro.py#L283

Translation: for all ASCII characters you have to answer with the same one random character, same as in the first task, but for the characters !":?.,; and space you have to answer the question character.

Looks like the first task can occur multiple times at the beginning before the second task is chosen. This task scheduler is used:

https://github.com/general-ai-challenge/Round1/blob/master/src/core/scheduler.py#L121

Edit 2: I checked the spec [PDF] (spoilers!) and the task described in section A.2.2 doesn't seem to match what I was seeing.

Looks like they have some problems with formatting in the PDF, the space character has not the same fixed width than the other characters. But it seems to be the second task as I described it.

But the tasks are just examples. They could use a complete different set for the non-public tasks when testing the AI programs.
So Long, and Thanks for All the Fish
Electronics, hiking, retro-computing, electronic music etc.: https://www.youtube.com/c/FrankBussProgrammer
 

Offline FrankBussTopic starter

  • Supporter
  • ****
  • Posts: 2365
  • Country: de
    • Frank Buss
Re: General AI challenge, $5 million prizes in total
« Reply #8 on: March 14, 2017, 12:54:27 pm »
The organizer updated the rules document:

https://discourse.general-ai-challenge.org/t/important-round-1-specifications-and-rules-update-1/298

Now it is more clear, that you can win the qualitative prize without even writing a single line of code, if you submit a nice white paper with some ideas.
So Long, and Thanks for All the Fish
Electronics, hiking, retro-computing, electronic music etc.: https://www.youtube.com/c/FrankBussProgrammer
 

Offline PointyOintment

  • Frequent Contributor
  • **
  • Posts: 327
  • Country: ca
  • ↑ I scanned my face
Re: General AI challenge, $5 million prizes in total
« Reply #9 on: March 16, 2017, 12:36:16 pm »
I refuse to use AD's LTspice or any other "free" software whose license agreement prohibits benchmarking it (which implies it's really bad) or publicly disclosing the existence of the agreement. Fortunately, I haven't agreed to that one, and those terms are public already.
 

Offline FrankBussTopic starter

  • Supporter
  • ****
  • Posts: 2365
  • Country: de
    • Frank Buss
Re: General AI challenge, $5 million prizes in total
« Reply #10 on: March 16, 2017, 02:26:22 pm »
Looks like DeepMind may have already won: http://www.wired.co.uk/article/deepmind-atari-learning-sequential-memory-ewc

The article says "While it is able to learn from its previous experiences and retain the most useful information, it isn't able to perform as well as a neural network that completes just one game.". But looks like their neural network dedicated to one game couldn't even complete an old Atari 2600 game:

http://www.wired.co.uk/article/google-ai-montezuma-revenge

Granted, most humans perform worse, but come on, 100 million tries and it still can just explore 15 out of 24 rooms? Remember, to solve the AI challenge, the program has to solve all tasks, they will just measure how many steps it needs. No chance.
So Long, and Thanks for All the Fish
Electronics, hiking, retro-computing, electronic music etc.: https://www.youtube.com/c/FrankBussProgrammer
 

Offline PointyOintment

  • Frequent Contributor
  • **
  • Posts: 327
  • Country: ca
  • ↑ I scanned my face
Re: General AI challenge, $5 million prizes in total
« Reply #11 on: March 16, 2017, 02:39:53 pm »
Isn't this challenge a lot simpler than even old Atari games? And the rules update post makes clear that even if one program does better in an absolute sense, what they're going to reward is learning and improving, even if it's slower overall.
I refuse to use AD's LTspice or any other "free" software whose license agreement prohibits benchmarking it (which implies it's really bad) or publicly disclosing the existence of the agreement. Fortunately, I haven't agreed to that one, and those terms are public already.
 

Offline FrankBussTopic starter

  • Supporter
  • ****
  • Posts: 2365
  • Country: de
    • Frank Buss
Re: General AI challenge, $5 million prizes in total
« Reply #12 on: March 16, 2017, 02:56:06 pm »
Maybe you are right. The source code for the Atari 2600 playing AI:

https://github.com/kuz/DeepMind-Atari-Deep-Q-Learner

Anyone who wants to try it with this challenge?
So Long, and Thanks for All the Fish
Electronics, hiking, retro-computing, electronic music etc.: https://www.youtube.com/c/FrankBussProgrammer
 

Offline FrankBussTopic starter

  • Supporter
  • ****
  • Posts: 2365
  • Country: de
    • Frank Buss
Re: General AI challenge, $5 million prizes in total
« Reply #13 on: March 16, 2017, 10:14:36 pm »
Nice, my webserver was featured in the weekly update:

https://www.general-ai-challenge.org/weekly-update-4
So Long, and Thanks for All the Fish
Electronics, hiking, retro-computing, electronic music etc.: https://www.youtube.com/c/FrankBussProgrammer
 

Offline FrankBussTopic starter

  • Supporter
  • ****
  • Posts: 2365
  • Country: de
    • Frank Buss
Re: General AI challenge, $5 million prizes in total
« Reply #14 on: November 01, 2017, 09:04:53 pm »
Just an update, the first round is finished. As expected: https://www.general-ai-challenge.org/gradual-learning

"None of the submitted agents were able to complete the evaluation curriculum. The most successful agent got close to completing 25% of the tasks (see the chart). Therefore, the quantitative prize was not awarded in this round."

It is like with these intelligence tests with number sequences. There are just too many possible continuations, and these tests had even more degrees of freedom. This might be the reason that there were only 13 submissions at all. Maybe the next round is more interesting: https://www.general-ai-challenge.org/ai-race But unless I missed it, they didn't say anything about what tasks has to be solved for it? :-//

So Long, and Thanks for All the Fish
Electronics, hiking, retro-computing, electronic music etc.: https://www.youtube.com/c/FrankBussProgrammer
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf