Thinking Complete

...you can download machine learning papers, which is almost the same thing, and I have so many papers on my to-read list. I like being able to write summaries of the most interesting ones, but when I hit 200 open tabs of interesting papers, I realised that the backlog was only going to keep piling up. So I've spent the day skimming as many as I could. There are many I'd like go to back and reread in more detail, but for now the summaries below will have to do. Deep learning theory Approximation by superpositions of a sigmoidal function . The original paper showing that sufficiently-wide one-hidden-layer neural networks can approximate any function. Why does unsupervised pre-training help deep learning?  Another classic paper, offering an explanation for why unsupervised pre-training improves performance: apparently it's an "unusual form of regularisation". (How widely is it still used?) Do deep nets really need to be deep?  Ba and Caruna show that shall

I just came back from a long weekend in Berlin, which I think of as the centre of Europe - not just geographically and historically, but also in its current role as the capital of Europe's most influential country. Although not as popular a destination as London or Paris, it's still a magnet for travellers - it felt like half the people around me on the streets were tourists, and I overheard conversations in English nearly as often as German (along with plenty of French, particularly at the screening of the World Cup final). I also wouldn't be surprised if the average Berliner I talked to were more proficient in English than the average Briton (although my sample was pretty skewed towards intellectuals). Coming from a country where even bilingualism is rare, I am continually impressed by European linguistic proficiency. And not just Europeans - during two recent trips to Morocco, I discovered that virtually all Moroccans are fluent in Arabic and French, plus a Berber diale

Dota 2 and Starcraft 2 are two of the most popular and complex esports; professional gamers train for years to master them. But how hard are they for AIs? DeepMind is working on the latter . OpenAI has already created a system ( OpenAI Five ) which is able to beat strong amateurs at a simplified version of the former, and they'll be challenging professionals in August. By their own account, they were surprised that their standard reinforcement learning approach worked so well, because of the game's complexity: Counting every fourth frame, as OpenAI Five does, Dota games last on average 20,000 "moves", compared with 40 for chess and 150 for go. Unlike chess and go, Dota is a partial-information game in which players need to infer what their enemies are doing. The range of actions in Dota is very large, with many of them varying continuously. Apparently, around 1000 are valid at each timestep (although I'm not quite sure what OpenAI are referring to as an actio