tag:blogger.com,1999:blog-3389115580319503562024-02-25T23:01:21.325-08:00For further thought...Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.comBlogger33125tag:blogger.com,1999:blog-338911558031950356.post-47490024918957151842016-10-20T11:53:00.002-07:002016-10-20T12:22:55.456-07:00ECCV in a theatrical setting<span style="color: #333333; font-family: "lato" , "helvetica neue" , "helvetica" , "helvetica" , "arial" , sans-serif; font-size: 14.4px; text-align: justify;"><a href="https://carre.nl/en/page/getting-here">Koninklijk Theater Carré (in Amsterdam),</a> where the main conference was held, is regularly used for theatrical and circus performances. The main stage was home to all the oral and most of the poster presentations during the week. This both meant that (i) speakers were performers with their audience looming above them from all sides and balconies, and that (ii) poster sessions from a birds-eye-view looked like a <a href="https://youtu.be/cDcprgWiQEY">simulation of particles</a> moving through a viscous liquid, trapped within the confines of the stage (scroll to the end of this post for a demonstration).</span><br />
<span style="color: #333333; font-family: "lato" , "helvetica neue" , "helvetica" , "helvetica" , "arial" , sans-serif; font-size: 14.4px; text-align: justify;"><br /></span>
<br />
<div style="text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNrJJSHWmTyfHNKjifkEusdsMANqg_eZQcj0zkgGIxyN0Ee5ib-h0BSXtHiJGLp-M4YqY2uGcjPEV9nLm1M-VJRsIRRM0Mm-L-QY2tI0-J6m9FtXWhbouWrtLH4bvh8bfzo6dw8JAACD3X/s1600/IMG_20161011_175520244.jpg" imageanchor="1"><img border="0" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNrJJSHWmTyfHNKjifkEusdsMANqg_eZQcj0zkgGIxyN0Ee5ib-h0BSXtHiJGLp-M4YqY2uGcjPEV9nLm1M-VJRsIRRM0Mm-L-QY2tI0-J6m9FtXWhbouWrtLH4bvh8bfzo6dw8JAACD3X/s400/IMG_20161011_175520244.jpg" width="400" /></a><span style="color: #333333; font-family: "lato" , "helvetica neue" , "helvetica" , "helvetica" , "arial" , sans-serif; font-size: 14.4px; text-align: justify;"><br /></span></div>
<br />
Due to this unusual set-up, audience questions could not be solicited in the usual manner of line ups in front of a microphone (try climbing over all those people, and out of a balcony). Instead, given a tech crowd, it was expected that technology could easily come to the rescue... the results of which can be summarized by comments made on separate occasions by the respective session chairs:<br />
<br />
<span id="docs-internal-guid-049dcde1-d99d-934e-6546-45ee01a0bff3"><span style="vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;"><i>"Please post your questions on twitter and we will ask them on your behalf [...] </i></span></span></span><br />
<span style="vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;"><i>But neither of us have twitter, so we will ask our own questions in this session."</i></span></span><br />
<i><span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;"><span style="vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: xx-small;"><br /></span></span>
</span></i><br />
<div style="text-align: right;">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;"><i><span style="vertical-align: baseline; white-space: pre-wrap;">"</span><span style="white-space: pre-wrap;">It seems the community is composed of two groups: </span></i></span></div>
<div style="text-align: right;">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;"><i><span style="font-size: xx-small;"><span style="white-space: pre-wrap;">those that have questions, </span></span><span style="font-size: xx-small;"><span style="white-space: pre-wrap;">and those that know how to use twitter </span></span></i></span></div>
<div style="text-align: right;">
<span style="font-size: xx-small;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small; white-space: pre-wrap;"><i>- we’re still hoping there will be an intersection at some point."</i></span></span></div>
<br />
There was little to complain about otherwise: the venue was quite beautiful; there were many comfortable corners all around the building that were quite favorable to getting some paper reading done; the little baked parmesan palmiers that waiters carried around on trays all throughout the day were impeccable; and the city surrounding the conference was bursting with energy and canals.<br />
<br />
<div style="text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZ1UOr_JRpim0uSjAIuZEbLwxJzrFMKmfv_zAFYDSOw6NjD5GMEd263-AOxo55vESTTe_yzC6pZtO-ILyT0-jWv3sAkkUiWLblSrzZNLzejggyAHE0-989KCF1QuMHcOmuHChKE7KQOh5c/s1600/IMG_20161010_110851839_HDR.jpg" imageanchor="1"><img border="0" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZ1UOr_JRpim0uSjAIuZEbLwxJzrFMKmfv_zAFYDSOw6NjD5GMEd263-AOxo55vESTTe_yzC6pZtO-ILyT0-jWv3sAkkUiWLblSrzZNLzejggyAHE0-989KCF1QuMHcOmuHChKE7KQOh5c/s400/IMG_20161010_110851839_HDR.jpg" width="400" /></a><br />
<br /></div>
<h4>
Main topics:</h4>
<div>
<span style="font-family: "times" , "times new roman" , serif;">During the welcome, the general chairs put up some statistics about the topic areas that represented ECCV this year. The top ones include:</span></div>
<span id="docs-internal-guid-049dcde1-dec1-4bca-1a89-ee4f6253e681"></span><br />
<ul style="margin-bottom: 0pt; margin-top: 0pt;"><span id="docs-internal-guid-049dcde1-dec1-4bca-1a89-ee4f6253e681"><span style="font-family: "times" , "times new roman" , serif;">
<li dir="ltr" style="font-size: 14.6667px; list-style-type: disc; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-size: 14.6667px; vertical-align: baseline; white-space: pre-wrap;">deep learning</span></div>
</li>
<li dir="ltr" style="font-size: 14.6667px; list-style-type: disc; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-size: 14.6667px; vertical-align: baseline; white-space: pre-wrap;">3D modeling</span></div>
</li>
<li dir="ltr" style="font-size: 14.6667px; list-style-type: disc; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-size: 14.6667px; vertical-align: baseline; white-space: pre-wrap;">events, actions</span></div>
</li>
<li dir="ltr" style="font-size: 14.6667px; list-style-type: disc; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-size: 14.6667px; vertical-align: baseline; white-space: pre-wrap;">object class detection</span></div>
</li>
<li dir="ltr" style="font-size: 14.6667px; list-style-type: disc; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-size: 14.6667px; vertical-align: baseline; white-space: pre-wrap;">semantic image</span></div>
</li>
<li dir="ltr" style="font-size: 14.6667px; list-style-type: disc; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-size: 14.6667px; vertical-align: baseline; white-space: pre-wrap;">object tracking</span></div>
</li>
<li dir="ltr" style="font-size: 14.6667px; list-style-type: disc; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-size: 14.6667px; vertical-align: baseline; white-space: pre-wrap;">de-blurring</span></div>
</li>
<li dir="ltr" style="font-size: 14.6667px; list-style-type: disc; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-size: 14.6667px; vertical-align: baseline; white-space: pre-wrap;">scene understanding</span></div>
</li>
<li dir="ltr" style="font-size: 14.6667px; list-style-type: disc; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-size: 14.6667px; vertical-align: baseline; white-space: pre-wrap;">image indexing</span></div>
</li>
<li dir="ltr" style="font-size: 14.6667px; list-style-type: disc; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-size: 14.6667px; vertical-align: baseline; white-space: pre-wrap;">face recognition</span></div>
</li>
<li dir="ltr" style="font-size: 14.6667px; list-style-type: disc; vertical-align: baseline;"><span style="font-size: 14.6667px; vertical-align: baseline; white-space: pre-wrap;">segmentation</span></li>
</span></span></ul>
<span id="docs-internal-guid-049dcde1-dec1-4bca-1a89-ee4f6253e681">
</span>
<br />
<div>
</div>
<span style="font-family: "times" , "times new roman" , serif;"><br /></span>
<span style="font-family: "times" , "times new roman" , serif;">Topics like sparse coding are going down in paper representation. High acceptance rate topics are confounded by the size of those topics: smaller topics have a larger relative percentage of that are accepted (e.g. model-based reconstruction, 3D representation, etc.). P<span style="white-space: pre-wrap;">opular reviewer subject areas mostly follow the top topic areas above - specifically: 3D modeling, deep learning, object class detection, events, face recognition, object class detection, scene understanding, etc. </span></span><br />
<span style="font-family: "times" , "times new roman" , serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "times" , "times new roman" , serif;"><span style="white-space: pre-wrap;"><b>Summary notes:</b></span></span><br />
<span style="font-family: "times" , "times new roman" , serif;"><span style="white-space: pre-wrap;"><b><br /></b></span></span>
<span style="font-family: "times" , "times new roman" , serif;"><span style="white-space: pre-wrap;">My summary notes on the presentations that I attended can be found here (covers ~70% of the oral sessions): <a href="https://docs.google.com/document/d/175ORVlLMdjOscJ7-93WIt0bieUiu21vtlL7J-7-7qBI/pub">https://docs.google.com/document/d/175ORVlLMdjOscJ7-93WIt0bieUiu21vtlL7J-7-7qBI/pub</a> </span></span><br />
<span style="font-family: "times" , "times new roman" , serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<br />
<h4>
Some general research trends*:</h4>
<div style="text-align: right;">
<span style="font-size: x-small;">* <b>disclaimer:</b> very biased by my own interests, observations, and opinions </span></div>
<div style="text-align: right;">
<span style="font-size: x-small;">(which tend to revolve around perception, cognition, attention, and language)</span></div>
<div style="text-align: right;">
<span style="font-size: x-small;">for an objective summary, go instead to the <a href="https://docs.google.com/document/d/175ORVlLMdjOscJ7-93WIt0bieUiu21vtlL7J-7-7qBI/pub">summary notes</a> linked above</span></div>
<br />
Nobody asks anymore: <i>"is this done with CNNs too?", </i>and<i> </i>more and more research is digging into the depths of the not-so-black* box of CNNs. The remaining fruits are now a little higher than they were before, and we are beginning to see more reaching - in the form of innovations in architectures, evaluations, internal representations, transfer learning, integration with new sensors/robotics, and unsupervised approaches. More about some of these below.<br />
<div style="text-align: right;">
<span style="font-family: "times" , "times new roman" , serif; font-size: xx-small;"><span style="font-family: "arial"; white-space: pre-wrap;">* With some notable exceptions -> C</span><span style="white-space: pre-wrap;">hair: “did you train with stochastic gradient descent?” Speaker: “we trained with caffe”</span></span></div>
<br />
We're seeing old ideas come back in new architectural forms: new ways of encoding long-thought-about constraints and relations. If one can open an old vision paper and reformulate the proposed pipeline as an end-to-end network, encode constraints and heuristics as appropriate loss functions, and leverage different task knowledge by designing a corresponding training procedure, then a new paper is in the making (e.g. active vision for recognition).<br />
<br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOVoUOYris0KyXLuvPRSG7HI7ner5rHqGM7fj5UUghd-feX4_UqkOiX-7QipQJNmBU9PzLdj8jpKAoQLUhBoDQ4WZjaRHx76XbNyxVFqFIiQADk_-ygybbpuwWKXr1A6WJAGCtkaisi2aF/s1600/Screen+Shot+2016-10-18+at+7.28.20+PM.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="135" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOVoUOYris0KyXLuvPRSG7HI7ner5rHqGM7fj5UUghd-feX4_UqkOiX-7QipQJNmBU9PzLdj8jpKAoQLUhBoDQ4WZjaRHx76XbNyxVFqFIiQADk_-ygybbpuwWKXr1A6WJAGCtkaisi2aF/s320/Screen+Shot+2016-10-18+at+7.28.20+PM.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">http://www.eccv2016.org/files/posters/S-1B-05.pdf</td></tr>
</tbody></table>
Themes that we are beginning to see more and more of: unsupervised learning, semi-supervised learning, and self-supervised learning (with varying degrees of overlap, depending on how you define them). The main idea being that with the deep and powerful architectures we have now, solving each new problem in an end-to-end fashion would require an Imagenet-scale dataset. Because this is not always possible, transferring knowledge, labels, and classifications across tasks, datasets, and individual frames/images is the sought-after approach.<br />
<br />
Video is a popular modality: temporal information can provide a strong supervisory signal for propagating labels across frames or for learning to do object detection from unlabeled video (e.g., <a href="http://www.eccv2016.org/files/posters/P-3B-39.pdf">Walker et al.</a>, <a href="http://www.eccv2016.org/files/posters/S-4A-06.pdf">Long et al.</a>). Key frames of an action or an event can serve as targets for the rest of the frames. For instance, <a href="http://www.eccv2016.org/files/posters/P-1B-31.pdf">Zhao et al.</a> perform facial expression recognition using peak facial expressions as a supervisory signal, by matching the internal representations (i.e. network features) of peak and non-peak facial expressions in order to build more robustness and invariance into the recognition pipeline. Similarly, photo sequences or collections provide loose temporal relationships that can be harnessed as a self-supervisory cue for predicting relevant/future photos (e.g, <a href="http://www.eccv2016.org/files/posters/P-3A-26.pdf">Sigurdsson, Chen & Gupta</a>). As a side note, there is a lot more work on multi-dimensional inputs (3D, video, image sequences/collections) than single-images. Even with single images, there is a lot more temporal processing (e.g., via attention modules, more about this below). In other words, tasks that can be summarized as "image in" -> "single-label prediction out" have pretty much been exhausted.<br />
<blockquote class="tr_bq">
<ul>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/P-3B-39.pdf">An Uncertain Future: Forecasting from Static Images using Variational Autoencoders</a>, <em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Jacob Walker, Carnegie Mellon University; Carl Doersch, Carnegie Mellon University; Abhinav Gupta, ; Martial Hebert, Carnegie Mellon University</em></span></li>
<li><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="border: 0px; font-style: normal; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.eccv2016.org/files/posters/S-4A-06.pdf">Learning Image Matching by Simply Watching Video</a>, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Gucan Long, NUDT; Laurent Kneip, Australian National University; Jose M. Alvarez, Data61 / CSIRO; Hongdong Li, ; Xiaohu Zhang, NUDT; Qifeng Yu, NUDT</em></span></em></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/P-1B-31.pdf">Peak-Piloted Deep Network for Facial Expression Recognition</a>, <em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Xiangyun Zhao, University of California, San Diego; Xiaodan Liang, Sun Yat-sen University; Luoqi Liu, Qihoo/360; Teng Li, Anhui University; Yugang Han, 360 AI Institute; Nuno Vasconcelos, ; Shuicheng Yan</em></span> </span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/P-3A-26.pdf" style="font-size: x-small;">Learning Visual Storylines with Skipping Recurrent Neural Networks</a><span style="font-size: xx-small;">, </span><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Gunnar Sigurdsson, Carnegie Mellon University; Xinlei Chen, CMU; Abhinav Gupta</em><span style="font-size: xx-small;"> </span></span></li>
</ul>
</blockquote>
Language is another powerful supervisory signal: images that share tags or words in their respective descriptions (think also: comments in the context of social media) can be used to train network representations to cluster such images closer together or further apart (e.g., <a href="http://www.eccv2016.org/files/posters/O-1B-03.pdf">Yang et al.</a>). Some further examples of self-supervision by language include the works of <a href="http://www.eccv2016.org/files/posters/O-1B-02.pdf">Rohrbach</a> and <a href="http://www.eccv2016.org/files/posters/O-1B-04.pdf">Lu</a>. Other examples of cues/tasks used as self-supervision to learn useful internal representations for other tasks: co-occurrence, denoising, colorization, sound, egomotion, context, and video. Ways of leveraging existing images, modifying them and then learning the mapping back to the original images can be used as free training data (e.g., colorization, discussed more below, or image scrambling: <a href="http://www.eccv2016.org/files/posters/P-4B-32.pdf">Noroozi & Favaro</a>).<br />
<blockquote class="tr_bq">
<ul>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/O-1B-03.pdf">Improving Multi-label Learning with Missing Labels by Structured Semantic Correlations</a>, <em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Hao Yang, NTU; Joey Tianyi Zhou, IHPC; Jianfei Cai, NTU</em></span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/O-1B-02.pdf">Grounding of Textual Phrases in Images by Reconstruction</a>, <em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Anna Rohrbach; Marcus Rohrbach, UC Berkeley; Ronghang Hu, UC Berkeley; Trevor Darrell, UC Berkeley; Bernt Schiele</em></span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/O-1B-04.pdf">Visual Relationship Detection with Language Priors</a>, <em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Cewu Lu, Stanford University; Ranjay Krishna, Stanford University; Michael Bernstein, Stanford University; Fei-Fei Li, Stanford University</em></span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/P-4B-32.pdf">Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles</a>, <em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Mehdi Noroozi, University of Bern; Paolo Favaro</em><span style="font-size: xx-small;"> </span></span></li>
</ul>
</blockquote>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3XfP4fTYuZBt673Lq1DCtkn3bX9lZzyZf43LNMiM0BwJ7eDoxlrI9FTaQnWnOLIvGt4uCVgc5g91BwD2JMFVgl6NL8X90_kc_S5s8FD2ZpqR3PTBcCVmLrGWEjadQ3ZUWla0Db9SxfUYB/s1600/Screen+Shot+2016-10-20+at+2.45.09+PM.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="101" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3XfP4fTYuZBt673Lq1DCtkn3bX9lZzyZf43LNMiM0BwJ7eDoxlrI9FTaQnWnOLIvGt4uCVgc5g91BwD2JMFVgl6NL8X90_kc_S5s8FD2ZpqR3PTBcCVmLrGWEjadQ3ZUWla0Db9SxfUYB/s400/Screen+Shot+2016-10-20+at+2.45.09+PM.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">http://www.eccv2016.org/files/posters/O-1B-04.pdf</td></tr>
</tbody></table>
Works that demonstrate new unsupervised approaches will typically do evaluation in one of the following ways: (i) show that useful intermediate features emerge, by visualizing what neurons learn to fire on (as in <a href="http://www.eccv2016.org/files/posters/O-1B-01.pdf">Owens</a>', <a href="http://www.eccv2016.org/files/posters/O-2B-03.pdf">Zhang</a>'s work, based on the approach introduced by <a href="https://people.csail.mit.edu/khosla/papers/iclr2015_zhou.pdf">Zhou et al.</a>), or (ii) show that the learned internal representation provides good initialization for other tasks - i.e. that it is amenable to transfer learning (see <a href="http://www.eccv2016.org/files/posters/O-3A-04.pdf">Larsson et al.</a> or <a href="http://www.eccv2016.org/files/posters/O-2B-03.pdf">Zhang's work</a> for more examples). A great example of this self-supervised learning approach is the work by <a href="http://www.eccv2016.org/files/posters/S-1B-05.pdf">Pinto et al.</a> who showed that a physical robot that grasped, pushed, and poked a whole bunch of objects a whole bunch of times could learn useful visual representations for other tasks. Demonstrating that a learned representation is useful can be done by fixing the network and using computed features to directly cluster/retrieve images, or learning a classifier on top of the computed features for a new task, or using the learned representation only as an initialization while retraining with new data. The latter approach is especially useful if the task for which the network needs to be retrained does not have enough training data for complete end-to-end learning, and the unsupervised approach can bootstrap some of the feature learning.<br />
<blockquote class="tr_bq">
<ul>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/O-1B-01.pdf" style="font-size: x-small;">Ambient sound provides supervision for visual learning</a><span style="font-size: xx-small;">, </span><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Andrew Owens, MIT; Jiajun Wu, MIT; Josh Mcdermott, MIT; Antonio Torralba, MIT; William Freeman, MIT</em></span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/O-2B-03.pdf" style="font-size: x-small;">Colorful Image Colorization</a><span style="font-size: xx-small;">, </span><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Richard Zhang, UC Berkeley; Phillip Isola, MIT; Alexei Efros</em></span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;"><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><span style="font-size: xx-small;"><span style="border: 0px; font-style: normal; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.eccv2016.org/files/posters/O-3A-04.pdf">Learning Representations for Automatic Colorization</a>, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Gustav Larsson, University of Chicago; Michael Maire, Toyota Technological Institute at Chicago; Greg Shakhnarovich, TTI Chicago, USA</em></span></em><span style="font-size: xx-small;"> </span></span></li>
<li><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="border: 0px; font-style: normal; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.eccv2016.org/files/posters/S-1B-05.pdf">The Curious Robot: Learning Visual Representations via Physical Interactions</a>, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Lerrel Pinto, Carnegie Mellon University; Dhiraj Gandhi, ; Yuanfeng Han, ; Yong-Lae Park, ; Abhinav Gupta</em></span></em></em></li>
</ul>
</blockquote>
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjfAaKiRNLCwyDOASE1KDYUemDwKmu9HiVahsU-WYpldzmihtWdolvkonLKIrTowqNrMxtOhakj1fHGzMxHTnG7kwr0RXNAPaJ39lxP5cstOwGJqZ385Knw1NDWMnT9obkVV09r6YsNrrNQ/s1600/Screen+Shot+2016-10-19+at+5.13.07+PM.png" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="148" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjfAaKiRNLCwyDOASE1KDYUemDwKmu9HiVahsU-WYpldzmihtWdolvkonLKIrTowqNrMxtOhakj1fHGzMxHTnG7kwr0RXNAPaJ39lxP5cstOwGJqZ385Knw1NDWMnT9obkVV09r6YsNrrNQ/s320/Screen+Shot+2016-10-19+at+5.13.07+PM.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">http://www.eccv2016.org/files/posters/O-4B-04.pdf</td></tr>
</tbody></table>
This also touches on an important trend: we are starting to see more integration with robotics. We are coming back to active vision (e.g., <a href="http://www.eccv2016.org/files/posters/O-4B-04.pdf">Jayaraman & Grauman</a>). New architectures and compute power are providing us with the capabilities of learning structure from (relatively unstructured) interactions. This area of research will likely see tremendous growth in the next few years.<br />
Deep is coming to a robotics lab near you.<br />
<blockquote class="tr_bq">
<ul>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/O-4B-04.pdf">Look-ahead before you leap: end-to-end active recognition by forecasting the effect of motion</a>, <em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Dinesh Jayaraman, UT Austin; Kristen Grauman, University of Texas at Austin</em></span></li>
</ul>
</blockquote>
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5Q0cLzWXjHUBuiWOjrNPAhHQouIJzuN304IrkLlasMkC1nuzoG9GOG0ZKGul3gsPTa6IoUuLjBHnKdwrDGM4uVg2TXC-FpPll-vG3pneGDm2xCiVDqW_QpHNG5BsJFBqIRMoXYJC_YKdZ/s1600/Screen+Shot+2016-10-20+at+2.49.27+PM.png" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="146" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5Q0cLzWXjHUBuiWOjrNPAhHQouIJzuN304IrkLlasMkC1nuzoG9GOG0ZKGul3gsPTa6IoUuLjBHnKdwrDGM4uVg2TXC-FpPll-vG3pneGDm2xCiVDqW_QpHNG5BsJFBqIRMoXYJC_YKdZ/s400/Screen+Shot+2016-10-20+at+2.49.27+PM.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">http://www.eccv2016.org/files/posters/P-3B-17.pdf</td></tr>
</tbody></table>
Language continues to be a hot topic. This includes image captioning (and variants, like <a href="http://www.eccv2016.org/files/posters/P-1B-42.pdf">Zeng's "title generation"</a>), and related tasks like visual question answering - VQA (e.g., <a href="http://www.eccv2016.org/files/posters/P-1A-25.pdf">Mallya & Lazebnik</a>, <a href="http://www.eccv2016.org/files/posters/P-1B-21.pdf">Lin & Parikh</a>), referring expressions (e.g., <a href="http://www.eccv2016.org/files/posters/S-1A-07.pdf">Hu et al.</a>, <a href="http://www.eccv2016.org/files/posters/S-1B-09.pdf">Yu et al.</a>), explanation generation (<a href="http://www.eccv2016.org/files/posters/P-2B-17.pdf">Hendricks et al.</a>), semantic tagging, and leveraging language as a supervisory cue for other visual recognition tasks (as discussed above). Attention modules are also beginning to pop up more frequently: here, "attention" is used to refer to a modulation of (visual) features - a reweighing of which features, at which spatial locations, are used most at a given timestep (e.g., <a href="http://www.eccv2016.org/files/posters/O-3A-02.pdf">Zhang et al.</a>). Often, attention modules go hand-in-hand with recurrent neural networks (RNNs, e.g., LSTMs) that can encode temporal relationships. In this case, processing of the visual input at one time step influences processing at the next time step. For instance, captioning systems may "attend" to different image regions in sequence, while generating caption words sequentially. VQA systems may use a similar iterative procedure to refine the location of the image that can provide an answer to the question or aid with localizing a referring expression (e.g. <a href="http://www.eccv2016.org/files/posters/O-1B-02.pdf">Rohrbach et al.</a>, <a href="http://www.eccv2016.org/files/posters/P-3B-17.pdf">Xu & Saenko</a>).<br />
<blockquote class="tr_bq">
<ul>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/P-1B-42.pdf" style="font-size: x-small;">Title Generation for User Generated Videos</a><span style="font-size: xx-small;">, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Kuo-Hao Zeng, National Tsing Hua University; Tseng-Hung Chen, National Tsing Hua University; Juan Carlos Niebles, Stanford University; Min Sun, National Tsing Hua University</em></span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/P-1A-25.pdf" style="font-size: x-small;">Learning Models for Actions and Person-Object Interactions with Transfer to Question Answering</a><span style="font-size: xx-small;">, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Arun Mallya, UIUC; Svetlana Lazebnik</em><span style="font-size: xx-small;"> </span></span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/P-1B-21.pdf" style="font-size: x-small;">Leveraging Visual Question Answering for Image-Caption Ranking</a><span style="font-size: xx-small;">, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Xiao Lin, Virginia Tech; Devi Parikh, Virginia Tech</em><span style="font-size: xx-small;"> </span></span></li>
<li><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="border: 0px; font-style: normal; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.eccv2016.org/files/posters/S-1A-07.pdf">Segmentation from Natural Language Expressions</a>, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Ronghang Hu, UC Berkeley; Marcus Rohrbach, UC Berkeley; Trevor Darrell</em></span></em></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/S-1B-09.pdf" style="font-size: x-small;">Modeling Context in Referring Expressions</a><span style="font-size: xx-small;">, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Licheng Yu, University of North Carolina; Patrick Poirson, ; Shang Yang, ; Alex Berg, ; Tamara Berg, University on North Carolina</em><span style="font-size: xx-small;"> </span></span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/P-2B-17.pdf" style="font-size: x-small;">Generating Visual Explanations</a><span style="font-size: xx-small;">, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Lisa Anne Hendricks, UC Berkeley; Zeynep Akata, ; Marcus Rohrbach, UC Berkeley; Jeff Donahue, UC Berkeley; Bernt Schiele, ; Trevor Darrell</em><span style="font-size: xx-small;"> </span></span></li>
<li><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="border: 0px; font-style: normal; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.eccv2016.org/files/posters/O-3A-02.pdf">Top-down Neural Attention by Excitation Backprop</a>, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Jianming Zhang; Zhe Lin, Adobe Systems, Inc.; Jonathan Brandt; Xiaohui Shen, Adobe; Stan Sclaroff, Boston University</em></span></em></em></li>
<li><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="border: 0px; font-style: normal; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.eccv2016.org/files/posters/O-1B-02.pdf">Grounding of Textual Phrases in Images by Reconstruction</a>, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Anna Rohrbach; Marcus Rohrbach, UC Berkeley; Ronghang Hu, UC Berkeley; Trevor Darrell, UC Berkeley; Bernt Schiele</em></span></em></em></em></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/P-3B-17.pdf" style="font-size: x-small;">Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering</a><span style="font-size: xx-small;">, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Huijuan Xu, UMass Lowell; Kate Saenko, University of Massachusetts Lowel</em></span></li>
</ul>
</blockquote>
In general, many more works are using RNNs - and this is because some portion of the input or required output can be interpreted as a sequence: e.g. a sequence of frames, a sequence of images in a collection, or a sequence of words (in the input question or output caption). RNNs have also been shown to provide effective iterative refinement (e.g. <a href="http://www.eccv2016.org/files/posters/S-1A-08.pdf">Liang et al.</a>). An "attention module" can similarly be used to parse an image or image features as a sequence (e.g. <a href="http://www.eccv2016.org/files/posters/O-1A-04.pdf">Xiao et al.</a>, <a href="http://www.eccv2016.org/files/posters/O-1A-03.pdf">Peng et al.</a>, <a href="http://www.eccv2016.org/files/posters/P-3C-19.pdf">Ye et al.</a>). What this accomplishes is some simulation of bottom-up combined with top-down reasoning. And by the way, we <span id="goog_1809091037"></span><a href="http://saliency.mit.edu/ECCVTutorial/towardsCognitiveSaliency_ECCV_notes.pdf#page=43">talked a bit<span id="goog_1809091038"></span> about attention</a> and how it can be used to leverage other vision tasks in our <a href="http://saliency.mit.edu/ECCVTutorial/ECCV_saliency.htm">Saturday tutorial</a>.<br />
<blockquote class="tr_bq">
<ul>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/S-1A-08.pdf" style="font-size: x-small;">Semantic Object Parsing with Graph LSTM</a><span style="font-size: xx-small;">, </span><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Xiaodan Liang, Sun Yat-sen University; Xiaohui Shen, Adobe; Jiashi Feng, NUS; Liang Lin, Sun Yat-sen University; Shuicheng Yan, NUS</em></span></li>
<li><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="border: 0px; font-style: normal; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.eccv2016.org/files/posters/O-1A-04.pdf">Robust Facial Landmark Detection via Recurrent Attentive-Refinement Networks</a>, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Shengtao Xiao, National University of Singapore; Jiashi Feng, NUS; Junliang Xing, Chinese Academy of Sciences; Hanjiang Lai, SUN YAT-SEN UNIVERSITY; Shuicheng Yan, National University of Singapore; Ashraf Kassim, National University of Singapore</em></span></em></li>
<li><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="border: 0px; font-style: normal; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.eccv2016.org/files/posters/O-1A-03.pdf">A Recurrent Encoder-Decoder Network for Sequential Face Alignment</a>, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Xi Peng, Rutgers University; Rogerio Feris, IBM Research Center, USA; Xiaoyu Wang, Snapchat Research; Dimitris Metaxas, Rutgers University</em></span></em></em></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/P-3C-19.pdf" style="font-size: x-small;">Spatial Attention Deep Net with Partial PSO for Hierarchical Hybrid Hand Pose Estimation</a><span style="font-size: xx-small;">, </span><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Qi Ye, ; Shanxin Yuan, Imperial College London; Tae-Kyun Kim, Imperial College London</em></span></li>
</ul>
</blockquote>
With regards to image understanding and language, beyond scene recognition and object detection, we are also seeing increasing interest in interaction and relationship detection (e.g. <a href="http://www.eccv2016.org/files/posters/P-1A-25.pdf">Mallya & Lazebnik</a>, <a href="http://www.eccv2016.org/files/posters/O-1B-04.pdf">Lu et al.</a>, <a href="http://www.eccv2016.org/files/posters/P-3A-17.pdf">Nagaraja et al.</a>). I also found quite interesting the applications of language to non-natural images - specifically, diagrams (<a href="https://arxiv.org/abs/1603.07396">Kembhavi et al.</a>, <a href="http://www.eccv2016.org/files/posters/P-3B-29.pdf">Siegel et al.</a>).<br />
<blockquote class="tr_bq">
<ul>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.eccv2016.org/files/posters/P-1A-25.pdf">Learning Models for Actions and Person-Object Interactions with Transfer to Question Answering</a>, </span><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Arun Mallya, UIUC; Svetlana Lazebnik</em></span></li>
<li><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="border: 0px; font-style: normal; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.eccv2016.org/files/posters/O-1B-04.pdf">Visual Relationship Detection with Language Priors</a>, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Cewu Lu, Stanford University; Ranjay Krishna, Stanford University; Michael Bernstein, Stanford University; Fei-Fei Li, Stanford University</em></span></em></li>
<li><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="border: 0px; font-style: normal; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.eccv2016.org/files/posters/P-3A-17.pdf">Modeling Context Between Objects for Referring Expression Understanding</a>, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Varun Nagaraja, University of Maryland; Vlad Morariu, University of Maryland; Larry Davis, University of Maryland</em></span></em></em></li>
<li><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="border: 0px; font-style: normal; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="https://arxiv.org/abs/1603.07396">A Diagram Is Worth A Dozen Images</a>, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Aniruddha Kembhavi, AI2; Michael Salvato, Allen Institute for Artificial; Eric Kolve, Allen Institute for AI; Minjoon Seo, University of Washington; Hannaneh Hajishirzi, University of Washington; Ali Farhadi, University of Washington</em></span></em></em></li>
<li><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="border: 0px; font-style: normal; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.eccv2016.org/files/posters/P-3B-29.pdf">FigureSeer: Parsing Result-Figures in Research Papers</a>, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Noah Siegel, ; Zachary Horvitz, ; Roie Levin, ; Santosh Kumar Divvala, Allen Institute for Artificial Intelligence; Ali Farhadi, University of Washington</em></span></em></em></em></li>
</ul>
</blockquote>
<blockquote class="tr_bq">
</blockquote>
We continue to see interesting innovations in neural network architectures - for instance, alternatives to convolution filters (<a href="http://www.eccv2016.org/files/posters/O-3A-03.pdf">Liu et al.</a>, <a href="http://www.eccv2016.org/files/posters/O-4B-03.pdf">Danelljan et al.</a>), integration of CRFs with NNs (<a href="http://www.eccv2016.org/files/posters/P-1B-37.pdf">Arnab et al.</a>, <a href="http://www.eccv2016.org/files/posters/P-1A-36.pdf">Gadde et al.</a>, <a href="http://www.eccv2016.org/files/posters/P-3B-14.pdf">Chandra & Kokkinos</a>), and nice tricks to facilitate training like stochastic depth (<a href="http://www.eccv2016.org/files/posters/S-3A-08.pdf">Huang et al.</a>), to mention just a few.<br />
<blockquote class="tr_bq">
<ul>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/O-3A-03.pdf" style="font-size: x-small;">Learning Recursive Filters for Low-Level Vision via a Hybrid Neural Network</a><span style="font-size: xx-small;">, </span><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Sifei Liu, UC Merced; Jinshan Pan, UC Merced; Ming-Hsuan Yang, UC Merced</em></span></li>
<li><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="border: 0px; font-style: normal; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.eccv2016.org/files/posters/O-4B-03.pdf">Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking</a>, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Martin Danelljan, Linköping University; Andreas Robinson, Linköping University; Fahad Khan, Linkoping University, Sweden; Michael Felsberg, Link_ping University</em></span></em></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/P-1B-37.pdf" style="font-size: x-small;">Higher Order Conditional Random Fields in Deep Neural Networks</a><span style="font-size: xx-small;">, </span><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Anurag Arnab, University of Oxford; Sadeep Jayasumana, University of Oxford; Shuai Zheng, University of Oxford; Philip Torr, Oxford University</em></span></li>
<li><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="border: 0px; font-style: normal; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.eccv2016.org/files/posters/P-1A-36.pdf">Superpixel Convolutional Networks using Bilateral Inceptions</a>, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Raghudeep Gadde, Ecole des Ponts Paris Tech; Varun Jampani, MPI-IS; Martin Kiefel, MPI for Intelligent Systems; Daniel Kappler, MPI Intelligent Systems; Peter Gehler</em></span></em></li>
<li><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="border: 0px; font-style: normal; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.eccv2016.org/files/posters/P-3B-14.pdf">Fast, Exact and Multi-Scale Inference for Semantic Image Segmentation with Deep Gaussian CRFs</a>, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Siddhartha Chandra, INRIA; Iasonas Kokkinos, INRIA</em></span></em></em></li>
<li><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="border: 0px; font-style: normal; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.eccv2016.org/files/posters/S-3A-08.pdf">Deep Networks with Stochastic Depth</a>, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Gao Huang, Cornell University; Yu Sun, Cornell University; Zhuang Liu, Tsinghua University; Daniel Sedra, Cornell University; Kilian Weinberger, Cornell University</em></span></em></em></li>
</ul>
</blockquote>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKexzxCqe5xQIktDOpu-M7XVr8JgLq3RdEIDaSO4N19nOzJZVY2X9gOMPzxO2lbP-lnmNnSWjR1ecXQbV1Yq7dUkIkknIvhVHtq5pNOsdUSASIc1ygZccoKcv4oiL366si-vHkKf8ZpgNg/s1600/Screen+Shot+2016-10-20+at+2.48.38+PM.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="112" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKexzxCqe5xQIktDOpu-M7XVr8JgLq3RdEIDaSO4N19nOzJZVY2X9gOMPzxO2lbP-lnmNnSWjR1ecXQbV1Yq7dUkIkknIvhVHtq5pNOsdUSASIc1ygZccoKcv4oiL366si-vHkKf8ZpgNg/s320/Screen+Shot+2016-10-20+at+2.48.38+PM.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">http://www.eccv2016.org/files/posters/S-3A-08.pdf</td></tr>
</tbody></table>
Among very specific topics with unproportionally many papers this year: 11 papers on person re-identification, 6 papers on object counting (5 of which use CNNs), 3 papers with colorization applications (<a href="http://www.eccv2016.org/files/posters/O-2B-03.pdf">Zhang</a>, <a href="http://www.eccv2016.org/files/posters/O-3A-04.pdf">Larsson</a>, <a href="http://www.eccv2016.org/files/posters/O-3A-03.pdf">Liu</a>), and over 20 papers on segmentation and variations on segmentation (like portrait or scene matting, e.g., <a href="http://www.eccv2016.org/files/posters/S-1A-06.pdf">Shen et al.</a>). For instance, there were many improvements in semantic segmentation, and some domain-specific (e.g. biomedical) segmentation approaches presented (e.g. <a href="http://www.eccv2016.org/files/posters/S-1A-09.pdf">Liu et al.</a>).<br />
<blockquote class="tr_bq">
<ul>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/O-2B-03.pdf" style="font-size: x-small;">Colorful Image Colorization</a><span style="font-size: xx-small;">, </span><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Richard Zhang, UC Berkeley; Phillip Isola, MIT; Alexei Efros</em></span></li>
<li><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="border: 0px; font-style: normal; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.eccv2016.org/files/posters/O-3A-04.pdf">Learning Representations for Automatic Colorization</a>, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Gustav Larsson, University of Chicago; Michael Maire, Toyota Technological Institute at Chicago; Greg Shakhnarovich, TTI Chicago, USA</em></span></em></li>
<li><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="border: 0px; font-style: normal; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.eccv2016.org/files/posters/O-3A-03.pdf">Learning Recursive Filters for Low-Level Vision via a Hybrid Neural Network</a>, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Sifei Liu, UC Merced; Jinshan Pan, UC Merced; Ming-Hsuan Yang, UC Merced</em></span></em></li>
<li><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><span style="border: 0px; font-style: normal; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.eccv2016.org/files/posters/S-1A-09.pdf">SSHMT: Semi-supervised Hierarchical Merge Tree for Electron Microscopy Image Segmentation</a>, </span><em style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Ting Liu, University of Utah; Miaomiao Zhang, MIT; Mehran Javanmardi, University of Utah; Nisha Ramesh, University of Utah; Tolga Tasdizen, University of Utah </em></span></em></em></li>
<li><span style="font-family: Arial, Helvetica, sans-serif; font-size: xx-small;"><a href="http://www.eccv2016.org/files/posters/S-1A-06.pdf" style="font-size: x-small;">Deep Automatic Portrait Matting</a><span style="font-size: xx-small;">, </span><em style="border: 0px; color: #373737; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Xiaoyong Shen, CUHK; Xin Tao, CUHK; Hongyun Gao, CUHK; Chao Zhou, ; Jiaya Jia, Chinese University of Hong Kong</em><span style="font-size: xx-small;"> </span></span></li>
</ul>
</blockquote>
<h4>
<span style="font-weight: normal;">Interestingly, none of the award-winning papers were about neural networks.</span></h4>
<div>
<span style="font-weight: normal;"><br /></span></div>
<h4>
The future of vision conferences?</h4>
<h4>
<div style="font-weight: normal;">
<br />
It is interesting to observe how fast this field evolves, and the impacts this has on researchers, research programs, the publishing pipeline, and the outcome of conferences. In particular, it is now common for papers to be hanging up on arxiv for over half a year before they are presented at a conference. Occasionally this can lead to confusion, with researchers scratching their heads, surprised to stumble upon a particular paper at the conference (<i>hasn't this paper already been published for a while? hasn't it already appeared in the mass media?</i>) By the time the conference rolls around, other researchers may already be familiar with the paper, and may have even innovated on top of it. </div>
<div style="font-weight: normal;">
<br /></div>
<div style="font-weight: normal;">
With the speed of innovation, at the same conference you might find both papers that build upon previous architectures to improve their pitfalls, and other papers that completely replace the original architectures with large performance gains. Small improvements are likely to be quickly overstepped by more significant leaps that leave the small improvements quickly forgotten. Lasting work requires qualitatively new approaches. </div>
<div style="font-weight: normal;">
<br /></div>
<div style="font-weight: normal;">
It was interesting to see that a number of researchers presented their original published results (from the camera ready version of the paper) alongside new results obtained since, in an attempt to stay current - after all, half a year of additional innovations can change many numbers. Some of these additional innovations are a result of building upon recently-arxived work. Some presenters even explicitly make reference to an extension of the presented work that is already available on arxiv or is published in another venue. </div>
<div style="font-weight: normal;">
<br /></div>
<div style="font-weight: normal;">
This might explain some of the proliferation of computer vision research to other conferences. To get innovations out fast enough for them to remain relevant, it might make sense to publish them in the nearest upcoming venue than to wait for the next computer vision conference to roll around. We're seeing related papers pop up in satellite workshops, and other conferences in machine learning, graphics, robotics, and language (take your favorite computer vision researcher and check which venues they've most recently published in). </div>
<div style="font-weight: normal;">
<br /></div>
<div style="font-weight: normal;">
It has become common to hear: <i>"This was state of the art at the time of submission... But since then, we have been surpassed by multiple methods"</i>.</div>
<div style="font-weight: normal;">
This leads to an interesting conundrum: arxived work is not peer-reviewed, but creeps into presentations of peer-reviewed work at conferences. This is one way that presented work is made more current and relevant. Is this a symptom of the progress in this field outrunning the current conference structure? In some other fields (physics, biology, neuroscience, etc.), conference presentations are based on submitted abstracts, and publications are disentangled from conferences. However, I don't believe there are precedents of a field moving this fast. This is a difficult question.</div>
<div style="font-weight: normal;">
<br /></div>
<div style="font-weight: normal;">
But on the topic of modernizing conferences, something needs to be done about the overcrowding situation around posters (especially with attendance growing considerably). It's quite hard to find a spot to stand in front of a poster presenter, within audible distance and without occlusion. Up in the balcony of the Theater Carre, filming the craziness below, I daydreamed of staying comfortably seated while flying a drone to a perfectly-selected location in full view of a desired poster and presenter. Perhaps that kind of swarm behavior could be much more efficiently optimized using some future conference logistics software ;) In the meantime, here's my birds-eye-view:</div>
<div style="font-weight: normal;">
<br /></div>
<div class="separator" style="clear: both; font-weight: normal; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/jtSROe9AxDs/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/jtSROe9AxDs?feature=player_embedded" width="320"></iframe></div>
<div style="font-weight: normal;">
<br /></div>
<div style="font-weight: normal;">
<br /></div>
<div style="font-weight: normal;">
<br /></div>
</h4>
Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com4tag:blogger.com,1999:blog-338911558031950356.post-38493202894168939162016-06-29T23:27:00.000-07:002016-06-29T23:27:55.597-07:00Diversifying bias: how dataset bias can hurt and what we can do about itA very important topic for consideration, the question of dataset bias has been getting into the mainstream more and more recently: e.g. <a href="https://www.technologyreview.com/s/601775/why-we-should-expect-algorithms-to-be-biased/">"Why we should expect algorithms to be biased"</a>, and<br />
<a href="http://www.nytimes.com/2016/06/26/opinion/sunday/artificial-intelligences-white-guy-problem.html?smid=fb-nytopinion&smtyp=cur&_r=0">"Artificial Intelligence's white guy problem"</a>.<br />
<br />
<a href="http://people.csail.mit.edu/torralba/publications/datasets_cvpr11.pdf">As a research curiosity, dataset bias</a> has been shown to affect model generalizability: a machine learning algorithm trained on one dataset, and receiving flying colors on a particular collection of test images, may have abysmal performance on a different dataset with different image statistics. You can think about this as the case of only ever seeing faces front and center in an image, and then being tested on off-center faces and realizing you are unexpectedly, but miserably, failing at detecting them. Some real-life examples: <a href="http://www.wired.com/2009/12/hp-notebooks-racist/">"HP investigates claims of racist computer"</a>, <a href="http://gizmodo.com/5256650/camera-misses-the-mark-on-racial-sensitivity">"Camera misses the mark on racial sensitivity</a>".<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhUAXlxx5C9fu8xSlmrnUoGiV2mhH2iZ-vqqwZ8OLUjmYsFJSCKLs6hd90j7QBMWcrms_OC2Mp9z3bYQzDj-eVdSaypZhP849w_wq1LbEjGKYTOwTzam0QIlxZSvlHjmkHCFT2PLi0mitN_/s1600/CV-Dazzle-antiface.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="222" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhUAXlxx5C9fu8xSlmrnUoGiV2mhH2iZ-vqqwZ8OLUjmYsFJSCKLs6hd90j7QBMWcrms_OC2Mp9z3bYQzDj-eVdSaypZhP849w_wq1LbEjGKYTOwTzam0QIlxZSvlHjmkHCFT2PLi0mitN_/s400/CV-Dazzle-antiface.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-size: xx-small;">http://www.selfieresearchers.com/wp-content/uploads/2014/09/CV-Dazzle-antiface.png</span></div>
<br />
<br />
A more relevant and pressing example concerns the self-driving car. Ultimately, if trained correctly, it will learn to avoid pedestrians, lamp posts, barriers, and whatever else was meticulously labeled and included into its training set. But will it know to avoid a kangaroo if the most it ever saw in its training data and prior experience is a deer or a cow? For an algorithm to be capable of this type of generalization is a reasonable expectation. So even though the car might not be able to accurately determine what is happily hopping across the street, it will guess that it is a bit of a deer and a bit of a cow and since it is upright, maybe a bit of a pedestrian as well... but overall, and most importantly, whatever its identity, this happy creature should not be run over.<br />
<i><br /></i>
<i>Speaking of cows...</i><br />
<i><br /></i>
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-8xCwgI9zd7Ke1f9KqefObZu4ljR6zcf6PmUlqFuTRCaUM_vZcXeEVEj5jbhmMot9jQ-tc77U0PBJ6B7QQnGv_AsiB9AS7Nsm83zfk-FktYTBRT3xY3g_zXHZbLnLHj7Zdp6zUPRB7OfJ/s1600/fitbit_for_cows.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="175" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-8xCwgI9zd7Ke1f9KqefObZu4ljR6zcf6PmUlqFuTRCaUM_vZcXeEVEj5jbhmMot9jQ-tc77U0PBJ6B7QQnGv_AsiB9AS7Nsm83zfk-FktYTBRT3xY3g_zXHZbLnLHj7Zdp6zUPRB7OfJ/s320/fitbit_for_cows.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="font-size: 12.8px; text-align: center;"><div>
<div class="" style="clear: both; font-size: medium;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-8xCwgI9zd7Ke1f9KqefObZu4ljR6zcf6PmUlqFuTRCaUM_vZcXeEVEj5jbhmMot9jQ-tc77U0PBJ6B7QQnGv_AsiB9AS7Nsm83zfk-FktYTBRT3xY3g_zXHZbLnLHj7Zdp6zUPRB7OfJ/s1600/fitbit_for_cows.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><span style="font-size: xx-small;">https://www.insidescience.org/content/fitbit-cows/3076</span></a></div>
</div>
</td></tr>
</tbody></table>
Let me make an aside here and say that the sole fact that our machine learning algorithms (the ones behind the self-driving cars, autonomous appliances and robots of the future) are increasingly relying on data (are "data-driven") and are increasingly likely to be neural networks (which happen to chug through and learn from large amounts of data very well) is not in itself a reason for concern. I do not buy the argument that we should fear our "black box algorithms" because they are parameter-bloated, connected and intertwined networks that are "hard to understand and harder to interpret". Until quite recently, when asked what computer vision was up to, I would sarcastically answer: <i>"detecting cows on fields... but only patched ones, on green fields, in the center of the image, and only if awake"</i>. We wouldn't even be having some of these conversations (in the media) about dataset bias if neural networks weren't performing this well (there are bigger problems at stake if even the cows can't be detected). Having large amounts of data and the architectural machinery to deal with it is precisely what is helping us learn and generalize better.<br />
<br />
With that aside, it is nevertheless crucial to start thinking more carefully about dataset bias and model generalization. This thinking should not pit us against data-driven algorithms in any way; rather, we should continue to remind ourselves that it is the great prediction potential of the algorithms that is granting us the opportunity to think about these questions in the first place.<br />
<br />
There is no doubt that our current, state-of-the-art models are suffering from bias in their datasets. Otherwise, Google wouldn't have made <a href="http://www.nytimes.com/2012/06/26/technology/in-a-big-network-of-computers-evidence-of-machine-learning.html">a headline that was centered around how well its algorithm could recognize cats</a> (as opposed to anything else, really) by learning from YouTube videos. DeepDream wouldn't be <a href="http://www.fastcodesign.com/3048941/why-googles-deep-dream-ai-hallucinates-in-dog-faces">imbuing every single photo with hallucinogenic dogs</a>. Microsoft's AI chatbot wouldn't have learned in the span of only a day that <a href="http://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist">being a racist asshole gets people's attention</a>. Researchers wouldn't be spending their time <a href="http://www.evolvingai.org/fooling">fooling the algorithms</a>. The list goes on.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/DgPaCWJL7XI/0.jpg" src="https://www.youtube.com/embed/DgPaCWJL7XI?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<br />
<br />
The problem is that this kind of dataset bias is unavoidable - because people are biased and they are the source of the data that we're feeding to our algorithms (and if we can learn anything from the above, it's that people seem to be biased towards putting others down on social media, but then compensating by flooding the net with pictures of cute animals). This means that, unwillingly, we may be imbuing our algorithms with negative qualities, behavior, and biases. We may be perpetuating biases that we would otherwise like to remove from society (see <a href="http://www.nytimes.com/2015/07/10/upshot/when-algorithms-discriminate.html">"When algorithms discriminate"</a>).<br />
<br />
And yet, when we try to actively interfere, <a href="http://www.wired.co.uk/article/instagram-pro-anorexia-search-terms">we can make the problem worse</a>. More frighteningly, if people know they can influence and have effects on the data, they may use this for their benefit, and this can have either positive or negative consequences for other members of society (as in this <a href="https://www.ted.com/talks/andreas_ekstrom_the_moral_bias_behind_your_search_results?language=en#t-373864">Ted Talk on the "Moral bias behind your search results"</a>). We are all responsible for the data that we put on the net (what we upload and what we search), and need to recognize that the biases that are out there are our own.<br />
<br />
But if the biases do get out there, should we get rid of them? What would "un-biasing" the data even mean? Who has the right to say that something should or should not appear in a search result? Would a top-down filtering of content, that would change the data everyone sees, even be appropriate? Most parents would disagree with a single, universal parental control for all of the world's children, and this is not much different. Different individuals, cultures, societies have different preferences and norms, different beliefs and taboos.<br />
<br />
Which brings me to the importance of diversity in the data that we have. I would not argue for artificially nudging the numbers or tweaking the data to try to eliminate certain kinds of biases, as this can have all sorts of secondary and unintended consequences (as the <a href="http://www.wired.co.uk/article/instagram-pro-anorexia-search-terms">Twitter example</a> has shown). Instead, the more people that are participating in the data that is being harnessed for training algorithms, the better. This naturally adjusts the data balance to more accurately reflect the population that is using it. One solution is just to <a href="https://www.technologyreview.com/s/544861/how-facebook-and-googles-plans-to-boost-internet-accessadvanced-in-2015/">put more people on the web</a>, and I think we will get there. Another is to bring more humanities folks into the tech loop. It would be great to have the perspective of anthropologists, sociologists, historians, policy and law makers, and psychologists for insights on cultural sensitivities, historical trends, crowd mentality, virality, societal pressures, etc. so that we can have better expectations of what the data may bring before its on our plates and we have to deal with the consequences. In this case, the suggested approach is to use this knowledge to adjust the data-collection procedures themselves rather than the data after-the-fact.<br />
<br />
If you were collecting a survey, a change in wording but not meaning might drastically change who would respond to it. How different cultures look at concepts like success, individuality, and norms are also crucially different and affect how and what they communicate about these topics. Take this simple example: say you collect perceptions about an exam from a group of schoolchildren. You get two answers: "it was easy! [secondary reason: I passed]", "it was ok [I think I got only 98%... where did the 2% go?]". Without knowing the context of the cultures, societies, or families from which these two responses came, you would have a very biased dataset (I'm reminded of <a href="http://gladwell.com/outliers/">Malcolm Gladwell's book Outliers</a>; or <a href="https://www.youtube.com/watch?v=a4TXS7ck8bQ">this talk</a>). And this extends beyond surveys. The behaviors you elicit (and end up collecting) from a group of users can depend crucially on how information, a task, or a UI are presented. Psychologists and sociologists know this very well. But they are also less likely (currently) to be the ones collecting the large datasets that modern-day computational algorithms are trained on.<br />
<br />
Questions of labeling are key. <i>What do you call that thing?</i> If you give it one label over another, a different set of properties or attributes might be retrieved. Consider an example: the labeling of street scenes. Here's a pedestrian, and another one. Here's a bicyclist. What about that person in a wheelchair? Is that a pedestrian or transportation device? How many body parts must be visible and moving for a pedestrian to be labeled as such? This labeling might affect how an algorithm analyzing the scene might predict the future movements of the participants and objects in the scene. This, in turn, might affect the decisions the algorithm (read: autonomous vehicle) makes.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjPF6pu0ZvnLSI3yNykaTuH5sBfgcfoXVs_vH76hBrXcx_XBZ0IwU18w7x2WZeRMCDrzo2M6TtJnBa1WUhtJyNJwp4-xI3HaQLLSQqChpkLWQq6LgqBUDkqlz6KM-IKEKpLOo8RABoUZ86S/s1600/psychedelic-city-pop-art-new-york-city-street-scene-miriam-danar.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjPF6pu0ZvnLSI3yNykaTuH5sBfgcfoXVs_vH76hBrXcx_XBZ0IwU18w7x2WZeRMCDrzo2M6TtJnBa1WUhtJyNJwp4-xI3HaQLLSQqChpkLWQq6LgqBUDkqlz6KM-IKEKpLOo8RABoUZ86S/s320/psychedelic-city-pop-art-new-york-city-street-scene-miriam-danar.jpg" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-size: xx-small;">http://fineartamerica.com/featured/psychedelic-city-pop-art-new-york-city-street-scene-miriam-danar.html</span></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both;">
Every labeler is biased. Biased by their culture, their society, and their experience. Instead of attempting to unbias the labels, we should introduce even more biased labelers... to compensate (<i>and please, let's throw something intellectual up on the net, at least once for every 100 cats...</i>). We should increase the diversity of the bias until, on average, we get something reasonable. Ensembles work. That's the <a href="https://www.ted.com/speakers/james_surowiecki">wisdom of crowds</a>. </div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
And what do we do in the meantime while we wait for the whole world's wisdom to accumulate on the net? We think harder about our data collection strategies and the tasks used; we spend more time debugging and visualizing the algorithms and the trends they pick up on; we consider how to present, display, and use the data; we brainstorm ways to annotate and make explicit whether certain labels, tags, or content are more likely to be controversial or subjective; and we treat predictions in this space with greater care, and importantly, less confidence. Just as we tend to dislike the individuals with the greatest bias but highest confidence, let's not fill our digital world with these personalities.</div>
<br />
<br />Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com0tag:blogger.com,1999:blog-338911558031950356.post-35203891867068356292016-02-27T11:11:00.000-08:002016-02-27T11:11:17.356-08:00My free business lesson from an Uber driver<div style="margin-bottom: 0in;">
<br />
</div>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;">Want
a free business class? Find out more about Uber (or try talking to
more Uber drivers). Every Uber driver has at least some kind of
opinion to share about the Uber business model (its upsides and
downsides for drivers and passengers alike), and some drivers (if you
are inquisitive) will provide you with additional business logistics.
Occasionally you will receive an outlook on the (potentially
sobering) present and future societal impacts of Uber. If you are
really lucky, you will have spent the whole ride entertained by a
detailed rundown of the business' history and how it has been
continuously adapting to novel locations, changing consumer demands,
and emerging competition. I found myself the lucky passenger of
precisely the latter kind of Uber driver on my last trip from SFO to
Palo Alto: a young latin American named Marcos, with a square jaw and
an equally square baseball cap bill. I will endeavor to provide a
recount of this conversation (really, a monologue punctuated by my
occasional requests for additional details). I aim also for my
recounting to have the properties of a conversation, in that
regardless of the factual accuracy of individual details or the exact
temporal sequence of events (potentially tainted by the knowledge of
my driver and my interpretation), the general outlines of the
high-level picture should nevertheless surface.</span></span></span></div>
<div style="margin-bottom: 0in;">
<br />
</div>
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="http://johnbracamontes.com/wp-content/uploads/2015/09/Uber-v-Taxi.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://johnbracamontes.com/wp-content/uploads/2015/09/Uber-v-Taxi.jpg" height="158" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><span style="font-size: xx-small;">source: http://johnbracamontes.com/</span></td></tr>
</tbody></table>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;">From
my driver Marcos I learned that Uber sprouted up in SF to fill an
existing gap in the market: the need for a professional and,
importantly, reliable chauffeuring service. An emerging
sentiment at the time was one of dissatisfaction with cab services,
passengers having to unwillingly deal with unreliable service and
rude or disrespectful drivers. The drivers seemed to have the
upper hand in this market and behaved accordingly. </span></span></span></div>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;">Although I am
not sure how widespread this sentiment was, I can attest to the fact
that this is the reason I've never liked taking cabs.</span></span></span></div>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;"></span></span></span><br /><span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;"></span></span></span></div>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;">Naturally,
this was the sort of inconvenience and unpleasantness that the more
financially-privileged were willing to pay to avoid. Uber saw this
opportunity, and was perfectly positioned to take advantage of it: in
a city with (1) a dense population packed in a relatively small
(drivable) area, (2) large and growing tech companies providing a
continuous supply of financially-privileged individuals, and (3) a
traditionally startup-friendly environment, where bold new ideas
regularly surface and are picked up by the wave of tech hype. And so,
Uber was born (in 2010). As a professional and reliable chauffeuring
service with a convenient mobile app interface (and up-to-date
updates on driver location), passengers would be picked up in shiny
black cars by courteous drivers in formal attire, offering additional
frills like water and mints for the on-the-run business man. Sure,
this was an expensive alternative to cabs, but to the users of
UberBLACK, it was well worth it. Behind the scenes the structure was
quite clever as well: Uber provided the cars and phones for the
drivers (equipped with Uber app and Google maps), and regular people
stepped up to provide themselves as drivers, no formal interviewing
procedure required. Marcos dropped out of his community college to
take on this new, respectable job that required a full-time
commitment. As the success of a personalized on-demand transportation
service had shown its colors, it opened up the market for new
variants. And this is where Lyft comes into the picture. </span></span></span></div>
<div style="margin-bottom: 0in;">
<br />
</div>
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; text-align: left;"><tbody>
<tr><td style="text-align: center;"><a href="https://tctechcrunch2011.files.wordpress.com/2014/08/lyft.jpg?w=738" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="202" src="https://tctechcrunch2011.files.wordpress.com/2014/08/lyft.jpg?w=738" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><span style="font-size: xx-small;">source: techcrunch.com</span></td></tr>
</tbody></table>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;">Lyft
aimed to capture another SF-based market segment: the young crowd of
current students and recent graduates, now employed at local
startups. An alternative transportation solution was needed for the
kind of person that ate Mexican from a food truck and sported
giveaway t-shirts acquired at hackathons and career fairs. Lyft
was marketed as an affordable ride-share, the distinctive pink
mustache on car bumpers trumpeting the friendly, hip, and easy-going
atmosphere that customers would learn to expect from it. In stark
contrast to UberBLACK, passengers would sit in the front, engage in
conversations with their casually-dressed drivers, and ride in
whatever car the driver happened to own. Whereas Uber lent its
drivers cars and phones, Lyft sent them giant pink furry mustaches.
The latter was more financially viable, allowing prices to drop to
student standards, well below cab fees. Importantly, Lyft drivers
could work on flexible schedules, squeezing in rides in the free
moments of the day, morning, evening, and between activities. Marcos
could now go back to college and pick up passengers in his free
time. </span></span></span></div>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;">Uber
wisely recognized that much of its infrastructure was already in
place to allow its service to be differentiated for different kinds
of customers. Uber then branched to provide a new option: UberX.
Learning from the successes and failures of the Lyft model, UberX
allowed drivers to work flexible hours in flexible attire, operating
their own vehicles – provided, and this is important, that the
vehicles passed some minimal quality standards (Lyft passengers had
begun to complain about the run-down condition of some of the cars).
The water and mints were still there. Drivers were encouraged to be
friendly and hip. </span></span></span></div>
<div style="margin-bottom: 0in;">
<br />
</div>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;">Uber
had a first-player advantage: it had been first in the market and
thus enough time to acquire a good reputation and loyal customers
through its UberBLACK service. UberX brought in new customers and
gave the old ones a flexible option. Provided with the same
reliability and courteousness, some of UberBLACK customers now opted
for the cheaper, more informal option. It is part of SF culture not
to flaunt financial well-being, as evidenced by the casual hoodies
and slacks regularly worn by some top tech executives. So black cars
became regular cars (that were nevertheless guaranteed not to be
run-downs). </span></span></span>
</div>
<div style="margin-bottom: 0in;">
<br />
</div>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;">As
an aside, Uber now has a variant that is intermediate between
UberBLACK and UberX. Do you want to get picked up by a
casually-dressed Uber driver but in a brand-name car like a BMW or
Mercedez for an intermediate price? Well now you can with Uber
Select. And if you don't want a fancy car to pull up at your office
entrance in SF, you can stick to UberX. Different Uber options happen
to be dominant in different cities. For instance, perhaps
unsurprisingly, LA tends to prefer the luxurious option.</span></span></span></div>
<div style="margin-bottom: 0in;">
<br />
</div>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;">After
the introduction of UberX, Uber's customer pool grew. This meant that
the density of ride requests was often higher on Uber than on Lyft.
Drivers had more customers overall and could cover smaller
distances between ride requests. Marcos and his friends signed
back on with Uber.</span></span></span></div>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;"><br /></span></span></span></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody>
<tr><td style="text-align: center;"><a href="https://tctechcrunch2011.files.wordpress.com/2014/08/rockem-uber-lyft1.jpg?w=600" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="179" src="https://tctechcrunch2011.files.wordpress.com/2014/08/rockem-uber-lyft1.jpg?w=600" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><span style="font-size: xx-small;">source: techcrunch.com</span></td></tr>
</tbody></table>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;">New
measures had to be taken. Lyft gave its drivers new incentives:
“complete X rides and receive a rebate on the hefty commissions
paid back to Lyft”. Uber followed <span style="color: black;"><span style="background: transparent;">suit</span></span>.
The new incentives served an additional purpose: having to complete a
minimum number of rides, many drivers could no longer afford enough
time to work for both companies and still complete enough rides with
each. Choices had to be made. Uber tried to give drivers incentives
for accepting all ride requests in a row. Drivers obliged and
accepted all that came their way. They accepted requests even if it
required going around the whole block just to pick up a passenger
directly on the opposite side of the street. Passenger wait times
increased. Passengers were not happy. Uber pivoted its incentives
structure.</span></span></span></div>
<div style="margin-bottom: 0in;">
<br />
</div>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;">A
vicious price war ensued. The water and mints disappeared from Uber
cars. With few noticeable differences between the two services
from the customer perspective, customers went where prices were
lower. Lower prices meant more ride requests and a quicker way to hit
the incentive ride minimum. Drivers went where there were more
customers. </span></span></span>
</div>
<div style="margin-bottom: 0in;">
<br />
</div>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;">As
Marcos prepared to drop me off in Palo Alto, he got his Lyft app
ready. He said he'd take the first request he got - Uber or Lyft.
Palo Alto has longer ride distances and fewer customers per square
area than SF. Time is costly, and Marcos would not spend it
passenger-less. After all, he needed to be in class soon. He let me
out. My half-hour, 21-mile ride cost $37.78, including a
$3.85 airport surcharge. Uber would take <span style="background: transparent;">20-30%</span>,
gas would cost Marcos another few dollars, and car depreciation isn't
to be forgotten either. Marcos told me that the prices are more
expensive in SF than surrounding areas. (In fact, my trip back to SFO
from Palo Alto 2 days later cost $28.47). On my Uber app, I gave
Marcos 5 stars and left some feedback about what a knowledgable
guide he turned out to be. Then again, I don't remember the last time
I gave a poor review.</span></span></span></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://images.cdn.stuff.tv/sites/stuff.tv/files/styles/big-image/public/best-apps-of-the-year-uber.jpg?itok=FT3qL5U2" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://images.cdn.stuff.tv/sites/stuff.tv/files/styles/big-image/public/best-apps-of-the-year-uber.jpg?itok=FT3qL5U2" height="300" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><span style="font-size: xx-small;">source: http://images.cdn.stuff.tv</span></td></tr>
</tbody></table>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;">Lowering
prices means even more burden on the drivers. Already the fraction of
a cab fee, Uber fees are reaching new lows. Two days later, I logged
onto my Uber app at 5 a.m. to request a car back to the airport. I
could see some cars circling around the Googleplex complex, 15
minutes from where I was. After about 2 minutes, an Uber driver
accepted my request. Another minute later, he canceled the request.
He'd probably gotten a more conveniently-located ride request and
would make more money by keeping the distance driven without
passengers minimal (and 15 minutes was already pushing it). His car
stayed around the Googleplex complex. I placed another request,
finding myself irritated that it was taking me longer than 5 minutes
to get a car. My last dozen or so Uber trips involved instantaneous
request acceptance, with a car picking me up 1-2 minutes later. How
spoiled I had become. Finally, after another 3 minutes, my request
was accepted by a middle-aged Latin American gentleman named Juan
Carlos, and in 15 minutes, he was at my hotel.</span></span></span></div>
<div style="margin-bottom: 0in;">
<br />
</div>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;">I
was really thankful to Juan for picking me up. He was surprised to
find out there wasn't a swarm of cars ready to take me. Uber cars
often outnumber passengers at this early time in the morning, he told
me. I was in turn surprised to hear this, having spent that night
tossing and turning in bed worried that no Uber drivers would be on
the roads so early (I didn't even consider cabs as an alternative
anymore). Our differing expectations for what would be the Uber
availability situation that morning led me to thinking that there are
too many variables at play to fully predict driver behavior. Uber
drivers have to somehow optimize ride fares, company incentive
structures, passenger availability, and competition with other Uber
cars to figure out if a particular ride is going to bring them more
than it will cost. Earlier that morning Juan had driven another
passenger to San Jose airport - a 20 minute ride that cost the
passenger $10, of which Juan would probably get less than<span style="background: transparent;">
$6-7</span>.</span></span></span></div>
<div style="margin-bottom: 0in;">
<br />
</div>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;">I
told Juan about one of my recent Uber experiences in Boston. I had
decided to try UberPOOL for the first time: a variant where multiple
passengers can share the same ride, with different initial and
destination locations, as long as the trips are relatively in the
same direction. Each passenger pays less in return for the
potentially longer ride. If multiple passengers are picked up, the
Uber driver can hope to make a sliver more in the same fraction of
time by combining the trips. The interesting catch is: you get a
guaranteed UberPOOL price regardless of whether another passenger is
taken. In other words, you pay a lower price (even lower than UberX)
just by agreeing to potentially share the ride. Talking to my other
friends in Boston, it is pretty common for no additional passenger to
show up. So my friend and I took an UberPOOL. We counted as a single
passenger (it would be the same price if only one of us was there),
but didn't end up picking up a third passenger on the way. Our ride
was 10 minutes from Downtown Boston across the bridge to East
Cambridge, and cost us a total of $6. Splitting it, each of us paid
$3, almost the price of a subway ticket, but with the walking
distance (from subway to house) cut from 15 minutes down to zero.</span></span></span></div>
<div style="margin-bottom: 0in;">
<br />
</div>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;">Who
takes the loss when no additional passenger request is made on
UberPOOL: the company or the driver? I asked Juan. Turns out, it's
the driver (in Lyft's case, the company pays the difference). So if
drivers are making so little money, how can Uber remain a viable
longterm business model? Without missing a beat, Juan replied that it
doesn't need to be viable for longer than a decade at the most.
"After all, Uber is building a fleet of self-driving cars. No
paid drivers will be needed." Juan paused. But there's a bigger
problem: Juan is concerned about the strawberry-picking robots that
are now working on farms day and night, 24 hours straight. Soon,
there'll be even more robotic farm hands. Juan's family back in South
America along with thousands of other people are going to be out of
the farm jobs that provided their livelihood. "What happens
then?"</span></span></span></div>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;">Juan
Carlos got some fraction of the $28.47 I paid via my Uber app, and 5
stars.</span></span></span></div>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif;"><span style="font-size: small;"><br /></span></span></span></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://www.econlife.com/wp-content/uploads/2015/04/grapes-vineyard-704x260.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://www.econlife.com/wp-content/uploads/2015/04/grapes-vineyard-704x260.jpg" height="118" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><span style="font-size: xx-small;">source: http://www.econlife.com</span></td></tr>
</tbody></table>
<div style="margin-bottom: 0in;">
<span style="font-size: x-small;"><br /></span></div>
<div style="margin-bottom: 0in;">
<span style="color: black;"><span style="font-family: Arial, sans-serif; font-size: x-small;">Further reading:</span></span></div>
<div style="margin-bottom: 0in;">
<span style="font-size: x-small;"><br /></span></div>
<div style="margin-bottom: 0in;">
<span style="font-size: x-small;"><span style="font-family: Arial, sans-serif;"><span style="color: black;">Dated
sequence of events in Uber's history:
<a href="http://techcrunch.com/gallery/a-brief-history-of-uber/slide/26/">http://techcrunch.com/gallery/a-brief-history-of-uber/slide/26/</a>
</span></span>
</span></div>
<div style="margin-bottom: 0in;">
<span style="font-size: x-small;"><span style="font-family: Arial, sans-serif;"><span style="color: black;">More
about the Uber-Lyft battle:
<a href="http://www.npr.org/sections/alltechconsidered/2016/01/18/463473462/is-uber-good-to-drivers-it-s-relative">http://www.npr.org/sections/alltechconsidered/2016/01/18/463473462/is-uber-good-to-drivers-it-s-relative</a>
</span></span>
</span></div>
<div style="margin-bottom: 0in;">
<span style="font-family: Arial, sans-serif;"><span style="color: black; font-size: x-small;">How
much Uber drivers make:
<a href="http://www.buzzfeed.com/johanabhuiyan/what-uber-drivers-really-make-according-to-their-pay-stubs#.ropJdNaxxo">http://www.buzzfeed.com/johanabhuiyan/what-uber-drivers-really-make-according-to-their-pay-stubs#.ropJdNaxxo</a>
(compared to Lyft:
<a href="http://www.fastcompany.com/3048563/fast-feed/this-is-how-much-uber-and-lyft-drivers-make-in-different-cities">http://www.fastcompany.com/3048563/fast-feed/this-is-how-much-uber-and-lyft-drivers-make-in-different-cities</a>)</span></span></div>
<div style="margin-bottom: 0in;">
<span style="font-size: x-small;"><span style="font-family: Arial, sans-serif;"><span style="color: black;">Different
types of Uber services explained:
<a href="http://www.ridesharingdriver.com/whats-the-difference-between-uberx-xl-uberplus-and-black-car/">http://www.ridesharingdriver.com/whats-the-difference-between-uberx-xl-uberplus-and-black-car/</a>
</span></span>
</span></div>
<div style="margin-bottom: 0in;">
<span style="font-size: x-small;"><span style="font-family: Arial, sans-serif;"><span style="color: black;">Uber's
pricing and incentives model:
<a href="https://newsroom.uber.com/guest-post-a-deeper-look-at-ubers-dynamic-pricing-model/">https://newsroom.uber.com/guest-post-a-deeper-look-at-ubers-dynamic-pricing-model/</a>
</span></span>
</span></div>
<br />
<div style="margin-bottom: 0in;">
<span style="font-family: Arial, sans-serif;"><span style="font-size: x-small;">Quora discussion about Uber VS Lyft:
<a href="https://www.quora.com/What-is-the-difference-between-Uber-and-Lyft-1">https://www.quora.com/What-is-the-difference-between-Uber-and-Lyft-1</a></span>
</span>
</div>
Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com3tag:blogger.com,1999:blog-338911558031950356.post-38191847690661234272016-02-13T17:55:00.000-08:002016-02-13T17:55:08.645-08:00On effective communication: because it matters.I've been thinking quite a bit recently about effective communication, party because there were 2 seminars last month at MIT about giving good talks (one by <a href="http://people.csail.mit.edu/phw/">Patrick Winston</a>, one by <a href="http://www.principiae.be/X0101.php">Jean-luc Doumont</a>), party because we recently published a paper about <a href="http://news.mit.edu/2015/how-make-better-infographic-visualizations-1105">what makes visualizations effective</a> (for communicating messages), and partly because I've been TA-ing a <a href="https://news.mit.edu/2016/learning-to-solve-superurop-0201">research course for undergraduates</a> (with a large communication component to it).<br />
<br />
I'll summarize here some notes from the talks I went to, as well as my own thoughts and insights. Though I'm sure I'll have lots more to say on this topic in the future.<br />
<br />
Patrick Winston started off <a href="https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&uact=8&ved=0ahUKEwjXj4u-2_XKAhXEdT4KHUgGA4YQtwIIJDAB&url=https%3A%2F%2Fvimeo.com%2F101543862&usg=AFQjCNHFe90b0weHhDQQHp93KCbepmYkGA&sig2=MWF08JMDQ5dzNcnBgj_dZQ">his talk</a> with the following statement*: <span style="color: orange;">"you (the researcher) will be judged first by your speaking, then by your writing, and finally by your ideas"</span>. This is a common phenomenon: a great communicator can sell you on the simplest ideas and make you see beauty in them; a poor communicator can obscure the most beautiful of ideas. Both examples regularly occur in lectures, in research talks, and in business presentations (but I'll focus on the researchers, here). It really is a shame when beautiful ideas don't come to light because the researchers behind them lack in explanatory artistry. It <i>is</i> an art, this whole communication business - which is why it is not commonly taught in a formal manner. Aside from the occasional seminar, the <a href="https://people.csail.mit.edu/fredo/student.html">occasional resource</a> exchanged among students, and the occasional tip given by one researcher to another during a practice talk, aspiring researchers (e.g. students) get no formal coaching and are told to "just do good work". Feedback and tips from advisors can be quite uneven, depending on the experience of the advisors themselves. (luckily, MIT professors are very good at selling their research, judging by the content on the front page of <a href="http://news.mit.edu/">MIT news</a> every morning; as Winston puts it: <span style="color: orange;">"your ideas should have the wrapping that they deserve"</span>)<br />
<br />
The point is: <span style="color: orange;">many (esp. young) researchers need formal communication coaching, and often they underestimate how important it is for their careers</span> <i>(it pains me to hear yet another graduate student proclaim: "boy, these talks and posters I have to present are such a waste of my time"*)</i>. I would like to applaud MIT's initiative: the new <a href="https://www.eecs.mit.edu/node/6271#overlay-context=node/6271">EECS communication lab</a> (and similar ones in other departments) for providing resources, training and advisors to students, when they need them. Additionally, I think MIT's <a href="https://news.mit.edu/2016/learning-to-solve-superurop-0201">SuperUROP course</a> for undergraduates is a super valuable experience (essentially a how-to guide to being a researcher), where alongside a year's worth of academic research, students practice and receive feedback on important communication skills: writing research abstracts, proposals, and papers; performing peer reviews, creating academic posters, and giving research pitches and presentations. And yes, as a TA in the course, I sometimes hear the same excuses <i>("boy, all these written assignments are such a waste of my time, why can't I just do the research"</i>). But when you're in an environment where industry representatives, senior researchers, and MIT faculty are following what you're doing (as is the case for these students), being able to sell your work can mean a lot for your future career. Last semester, for instance, the students participated in a large <a href="https://www.youtube.com/watch?v=pdm-HWo5Flk">poster session</a>, where they presented their work to all the aforementioned parties. I gathered some advice, common mistakes, and helpful suggestions in the <a href="http://web.mit.edu/zoya/www/posterRecommendations.pdf">linked-to set of slides</a>.<br />
<br />
<span style="font-size: x-small;">* Yes, yes, groundbreaking ideas can speak for themselves, but I guarantee that <i>most</i> ideas need someone speaking for them (at least to get them off the ground).</span><br />
<br />
Note that from one set of communication-related slides to another, from one talk to the next, the same kind of advice surfaces again and again. Most often, the views and suggestions presented are not idiosyncratic, but common, accepted, guidelines. <span style="color: orange;">We've all been in the audience: we know what catches our interest and what bores us to death (and it's often not the content to blame).</span><br />
<br />
Let me summarize (and paraphrase in my own words) some of Winston's talk advice:<br />
<br />
<ul>
<li><span style="color: orange;">start with an empowerment promise</span>: give your audience a sense that they will walk away with something (e.g. some newfound knowledge or ideas) from your talk, so they know what to look forward to and why they should care</li>
<li><span style="color: orange;">get your idea out quickly</span>, and cycle back: don't expect that all your audience members will follow along with you until the end, and do not leave the most important to last ("avoid the crescendo, just blurt it out"); come back to, and reinforce your points</li>
<li><span style="color: orange;">use verbal punctuation</span>: people fog out, so bring them back once in a while, especially to accentuate a switching of topics, slides, etc. (kind of like an <i>"ehem, you can wake up now, even if you've missed the last few minutes, I'm starting a new thread..."</i>); </li>
<li><span style="color: orange;">avoid near-misses:</span> foresee what the audience could be confused about and clarify your contributions</li>
<li><span style="color: orange;">what you end with is the last impression</span>: make it count, clarify your contributions, show your audience what they're walking away with; and remember: the final slide will be there forever, "don't squander this real estate" (<i>is your final slide the infamous and content-less "thank you"?</i>)</li>
<li>whether a poster or a presentation, what should come clearly through are <span style="color: orange;">your vision, steps, and contributions </span>(Winston even advocates naming the relevant sections/slides accordingly)</li>
</ul>
<div>
When approached once by a young researcher looking to get advice on his job-talk slides, Winston proclaimed: <span style="color: orange;">"too many slides and too many words"</span>. </div>
<div>
"How do you know?" the researcher asked.</div>
<div>
"It's almost universally true."</div>
<div>
<span style="font-size: x-small;">(Winston later added that allowing powerpoint to have less than 30-point font is probably Bill Gates' biggest fault. When text has to shrink that much, there is too much of it on the slides.)</span></div>
<div>
<br /></div>
<div>
This is the kind of advice that will come up again and again. People have the tendency to cram as much as possible into very small (spatial or temporal) frames. Researchers want to talk about <u>all</u> the great work they've done (not realizing that they're drowning out the most important parts). Students put all the details of their projects on their posters (not realizing that the contributions get lost). Here's my suggestion: do one pass of the content from which you want to pull slides/talking points, and extract the most important points. Sleep on it. Then pick the most important points out of your selection, and scrap the rest. Repeat. <span style="color: orange;">With enough cycles, you would have cleaned away the debris, exposing the shine of the main ideas.</span></div>
<div>
<br /></div>
<div>
What I like about Winston's communication advice is that he comes at it from the perspective of a scientist (he is, after all, a computer science professor at MIT). Sprinkled throughout his talk are technical references and examples. Most of all, he emphasizes <span style="color: orange;">the importance of projections</span> - the way an idea or a piece of work is communicated to an audience: the context, the stance, the voice, the presentation style, all of it.</div>
<div>
<br /></div>
<div>
Another individual with a great technical take on communication advice is <a href="http://www.principiae.be/X0101.php">Jean-luc Doumont</a> (got a physics PhD from Stanford). Jean-luc (<i>he prefers to be called by his first name</i>) consistently refers to the <span style="color: orange;">importance of increasing signal and eliminating noise</span> in a presentation, whether visual or oral. This concept is ever-present in his book: <a href="http://www.treesmapsandtheorems.com/">Trees, Maps, and Theorems</a> - which I highly recommend. </div>
<div>
<br /></div>
<div>
Note that "noise" can refer to many things at once. In the case of presentations, the noise is everything that is tangential to your main points - it is the 'ums' and 'likes' in your speech, the nervous pacing and awkward hand fidgeting, the excessive details on your slides (<i>do you really need your institute's logo on every slide?)</i>. In the technical writing, noise includes all the superfluous words (why say it in 10 words when you can say it in 3? why talk like a politician?).</div>
<div>
<br /></div>
<div>
With regards to maximizing signal, Jean-luc also talks about <span style="color: orange;">maximizing effective redundancy</span> - which is to say helping to carry the message across despite noisy channels (those you have no control over, like the audience's attention or knowledge; whereas noisy channels that you <i>do </i>have control over should be minimized). Redundancy can be verbal or nonverbal. It can be complementary. For instance, your slides could contain your main points, but you're also there to describe them. If someone misses it in your speech, they see it on the screen*. You can also get the important messages across again later, in the same or different words (<i>remember the cycling that Winston referred to?</i>).</div>
<div>
<br /></div>
<div>
<span style="font-size: x-small;">* This does </span><i style="font-size: small;">not </i><span style="font-size: x-small;">mean that what is on the screen should be what is said. The slides complement, not replace, the oral presentation. If people are spending all their cognitive resources reading your slides, they'll fail to process what you're saying, and that is where the communication breaks down.</span><span style="font-size: x-small;"> </span></div>
<div>
<br /></div>
<div>
Jean-luc's three laws to optimize communication are:</div>
<div>
<ul>
<li>first law: adapt to your audience</li>
<li>second law: maximize the signal to noise ratio</li>
<li>third law: use effective redundancy</li>
</ul>
<div>
(but remember: second law > third law)</div>
</div>
<div>
<br /></div>
<div>
When studying information visualizations (graphs, charts, plots, etc.), our <a href="http://vcg.seas.harvard.edu/files/pfister/files/infovis_submission251-camera.pdf">research team</a> also found that when given visualizations with redundant encodings - i.e. when the message was presented in a number of ways (as a trend line, as an annotation of the trend line, as a description of the plot, in the title, etc.), human observers were more likely to recall the message correctly (different people might need to see things presented in a different way). Conversely, too many extra details, unrelated visuals, or metaphors led to worse recall and confusion, in that observers might recall only a piece of the main message, or misremember it entirely. The take-away? Make your priority getting the signal across, scrap the rest. You can do so quite effectively using the title. Importantly, if your title contains your message, more observers will remember and recall it. </div>
<div>
<br /></div>
<div>
Here's a little piece of advice that also tends to repeat: <span style="color: orange;">make your titles count</span>. Be it the titles of talks, slides, section headings, visualizations/graphs. Jean-luc places a lot of emphasis on this in <a href="http://www.treesmapsandtheorems.com/">Trees, Maps, and Theorems</a>. He gives great examples of how scientists often caption their plots something like "Y as a function of X", where it is clear that what is plotted is, by no surprise: Y as a function of X. You haven't told the reader anything new or useful! Consider instead <span style="color: orange;">using this valuable real estate to convey the message of the plot</span>, such as "Y peaks when X is at its lowest value due to the effect of...". After hearing all of Jean-luc's examples of the way scientists title their slides, figures, etc., I got to thinking. It's true, they do! </div>
<div>
<br /></div>
<div>
I have since tried to be extra careful about my captions, my titles, my paper section headings, even my e-mail subject lines (I guess the current generations get a lot of twitter practice). I try to limit the noise, to imbue as much of the written text with meaning as possible, to carry across the most important points. In fact, when writing <a href="http://web.mit.edu/zoya/www/docs/main.pdf">my master's thesis</a>, I wanted the essence of the whole thesis to come through the list of contents, figures, and tables. I wanted the reader to walk away with the outlines of the story without even getting to the introduction. </div>
<div>
<br /></div>
<div>
Importantly, if the message can come across simply and quickly, <i>that is not a bad thing</i>. If there's an easier way to say something, why not say it? Jean-luc had great anecdotes at his lecture on "<a href="https://www.youtube.com/watch?v=IFu3jaLmse0&feature=youtu.be">Communicating science to nonscientists</a>" about how unnecessarily jargon-filled scientific communication can be. Here are a few of my favorite anecdotes (again, paraphrased):</div>
<div>
<ul>
<li>After a room full of experts took turns describing their own research topics to each other, they were asked: how many of those descriptions did you understand? Less than half. How many do you still remember? Maybe a few. And this is a room of scientists! Moreover, they consider this normal. How many talks do you remember from your last conference? How many were engaging from start to finish? (maybe... 1?)</li>
<li>When researchers are asked to describe what it is they do, and when they get to any specialized vocabulary, they tend to say it faster and to lower their voice. It is like they are trying to limit us the pain of trying to understand them by saying it fast and low. But that is exactly the opposite of what we need in order to understand! </li>
<li>A student shows Jean-luc a passage he has written. Jean-luc looks confused and asks the student to explain what he meant to say in the passage. The student says: "Well what I mean to say is [blabla]... but I just don't know how to say it." Well in the [blabla] was exactly the explanation!</li>
</ul>
<div>
Jean-luc advises scientists "not to write complicated out of the principle of revenge" (for other scientists who write this way). <span style="color: orange;">Do not try to prove to the whole world how complicated your research is. </span>Define technical words, avoid jargon, avoid synonyms, write simply. Provide reference points, comparisons, and examples. <span style="color: orange;">Give the why before the what</span>.</div>
</div>
<br />
<br />
I'll leave you on my favorite Jean-luc quote from <a href="http://www.treesmapsandtheorems.com/">Trees, Maps, and Theorems</a>: <span style="color: orange;">"Effective communication is getting messages across. Thus it implies someone else: it is about an audience, and it suggests that we get this audience to understand something. To ensure that they understand it, we must first get them to pay attention. In turn, getting them to understand is usually nothing but a means to an end: we may want them to remember the material communicated, by convinced of it, or ultimately, act or at least be able to act on the basis of it."</span><br />
<br />
And getting messages across first and foremost requires caring about the importance of getting those messages across. It is about recognizing and believing that effective communication matters. It is about adjusting your habits, your jargon, the amount of content on your slides, your projection, your figure captions and titles, and most importantly your awareness of all these things. Happy communicating!<br />
<br />
<br />Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com2tag:blogger.com,1999:blog-338911558031950356.post-24789821765269672132016-01-19T21:43:00.001-08:002016-01-19T21:43:40.701-08:00Self-driving cars, internet balloons, and why Google is radicalWent to a talk about <b>[x]</b> today (formerly: Google[x]).<br />
<br />
Unlike other Silicon Valley companies that are "making the world a better place" (according to Silicon Valley, the TV series), [x] is "making the world a <i>radically</i> better place" (according to today's presenter).<br />
<br />
If you followed Google I/O this year or gone to some other Google talk about [x] and its moonshots, or if you've just glued yourself to the aforementioned TV series, in all three cases you would have seen the following slide:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghwio8i_bUZSmzVv8rv2TDX5MTV7bFtiGt6a-6ZuqetQLyLJlN8V73ENMWmgSLeNPa9Fa_WTG26LS1WkTGHxXCCx0nZ4E-C_GMZQn9WODJH5jqJv48bBVDf8BXcUGRL3OVwiAxu77T6Mmc/s1600/googlex.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="213" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghwio8i_bUZSmzVv8rv2TDX5MTV7bFtiGt6a-6ZuqetQLyLJlN8V73ENMWmgSLeNPa9Fa_WTG26LS1WkTGHxXCCx0nZ4E-C_GMZQn9WODJH5jqJv48bBVDf8BXcUGRL3OVwiAxu77T6Mmc/s320/googlex.jpg" width="320" /></a></div>
<br />
At MIT today, we heard about 2 projects from [x] (out of the <a href="http://www.eweek.com/cloud/slideshows/10-bold-google-x-projects-aiming-for-tech-breakthroughs.html">10 or so</a> that are currently public): <b>self-driving cars</b> and <b>internet balloons</b>.<br />
<br />
-----<br />
<br />
<i>A huge problem?</i> The 1.2 million annual traffic accidents, 93% of which are due to human error.<br />
<i>The radical solution:</i> <a href="http://www.makeuseof.com/tag/how-self-driving-cars-work-the-nuts-and-bolts-behind-googles-autonomous-car-program/">self-driving cars</a> and the required road infrastructure.<br />
<i>Breakthrough tech:</i> software with realtime sensor processing.<br />
<br />
The principle on which this whole autonomous car thing hinges, is an initial full laser mapping of the urban area in which the car is to drive (road and buildings and all) with which the car's real-time sensor data is then aligned for accurate positioning and localization within the lane.<br />
<br />
<i>Is it feasible to have to pre-map every urban environment, and then to update it when it changes?</i> Ah, well on the one hand Google Streetview cars have already shown us something about feasibility, but in the longterm, the self-driving cars will continue to collect data as they drive, driving and mapping simultaneously.<br />
<br />
In fact, already these self-driving cars can send alerts to the other self-driving cars on the road when things look different than expected (<i>lane closures on this street, sending you an updated map...</i>). Such alerts also get sent to the home base for potential updates to the whole system. The cars are connected to each other and their home base via the internet. (<i>Is this information transfer secure? </i>Not to worry, Google knows a thing or two about internet security.)<br />
<br />
So, some basic rules are hard-coded, but there's also a lot of machine learning from the many hours spent by these cars on the roads in all sorts of situations. These cars have to learn about the distribution of pedestrian movements (how fast they walk, how quickly they can switch direction, etc.), the typical behaviors of pedestrians, bicyclists, and pedestrians in response to bicyclists. They plot out the trajectories of all vehicles currently on the road and anticipate their next move.<br />
<br />
<i>The big challenge? </i>Achieving ridiculous recall and precision. A recall of 99% when pedestrians are involved is not going to do it (<i>you just can't afford to lose a few pedestrians here and there while you tweak your algorithm</i>). Recall is very much about the safety of the pedestrians, but precision is also about the safety of the vehicles (and their passengers). If the car freaks out at every road sign blowing in the wind, not only will the ride be very uncomfortably jerky, but the car might swerve into other cars to avoid hitting the mistakenly classified "pedestrian".<br />
<br />
There's other behaviors built-in for the comfort and safety of the passenger: for instance, shifting in the lane (all while avoiding other cars) when passing large trucks. Even if you have everything under control, you don't want your passenger getting antsy about a truck that's too close for comfort.<br />
<br />
These cars also slow down at railroads, near parked cars, and while passing bicyclists. Their long-range and short-range sensors ensure the car is <i>very</i> much aware of its surroundings. So much so that the 15 cm resolution of its sensors allows the cars to recognize the fine hand gestures of the traffic controller waving in the middle of the intersection or the bicyclist signaling to change lanes. In making decisions, the cars also make use of all sorts of contextual information: are other cars moving? Why have they stopped? Are there traffic cones around?<br />
<br />
And all of this computation and communication is happening on a <i>single CPU</i>. How's that for efficient resource sharing? (<i>but watch out for GPUs coming to a car near you...</i>)<br />
<br />
These cars have been designed for zero driver assistance. Are you going to see any sort of control device built into them like a wheel or a break pedal? No chance. This is Google's approach. No need so far: of the 13 driverless car incidents to date, all were the fault of the other drivers. <i>(Side thought: what if the sheer sight of this car on the road is distracting?)</i><br />
<i><br /></i>
But these cars sure go through a lot of situational testing. And yes, they're good in harsh weather conditions too (confirmed by hardware reliability tests and buckets of water). The QA testing position for the self-driving car project must be damn awesome.<br />
<br />
-----<br />
<br />
<i>Another huge problem? </i>2/3 of the world does not have internet.<br />
<i>The radical solution? </i>Balloons!<br />
<i>Breakthrough tech: </i>large-scale dynamic optimization of balloon network.<br />
<br />
We're talking global<i> </i>optimization (<i>literally, global</i>). Consider a network of balloons that are distributed around the world that need to follow pre-planned flight paths, adapt to changing wind conditions, and deal with intermittent (sometimes flaky) instructions - all while providing continuity of internet service. This is <a href="http://www.wired.com/2013/08/googlex-project-loon/">Project Loon</a>.<br />
<br />
Communication with these balloons as they pass over the oceans is through satellite phones. In these conditions, instructions can be dropped, intermittent, or conflicting, and the balloons must nevertheless make good decisions based on limited information and changing wind gusts.<br />
<br />
<i>So how does it all work?</i> These balloons fly at an altitude of 20 km - twice as high as airplanes and the weather, so at least a few less problems to deal with. They follow air currents at different altitudes, and steer with vertical motion to end up in an air current moving in the desired direction. An internal balloon pumps air in and out, and with essentially the power of a fan, can move the exterior balloon up and down. Additional power comes from solar cells, but in most cases the wind currents are sufficient to propel the balloons.<br />
<br />
A network of balloons thus moves through air currents, one displacing another, to provide continuous, seamless internet service to the cities below. It's kind of like how when you're moving, your service has to remain continuous despite shifting cell towers; but in this case, the city below is stationary, and it is the internet source that is moving above. This is the local optimization part.<br />
<br />
Sometimes, balloons also need to be dispatched to the location of a natural disaster, and this has to happen <i>fast</i>. Balloons also need to function in all kinds of harsh conditions, and with local repair most often unavailable, redundancy is key. <i>Redundancy, redundancy, redundancy.</i> Remember how the self-driving cars had 1 CPU? Well these babies have upwards of 40. And if something does goes down, you have to go fetch it... wherever it ends up (<i>can you climb trees and mountains?</i>). Another damn awesome job.<br />
<br />
These projects, and all the rest in the [x] repository are driven in part by the <a href="http://money.cnn.com/2015/03/18/technology/google-x-astro-teller-sxsw/">slogan</a>: <i>"we need to fail faster". </i>Innovation comes from trying radically new things, and radically new things can often lead to failure. Failing faster means trying again sooner.<br />
<br />
I gotta hand it to you, Google sells it well. Another take-away? It seems Google likes the word <i>radical</i>.<br />
<br />
<br />Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com0tag:blogger.com,1999:blog-338911558031950356.post-3804434384009935732016-01-17T19:34:00.000-08:002016-02-03T09:05:55.045-08:00Pulling together the efforts of neuroscientists and computer scientists<br />
Had a great time last Friday at the intersection of Neuroscience and Computer Science: http://cbmm.mit.edu/science-engineering-vassar<br />
Heard from 6 MIT Superstars: <a href="https://billf.mit.edu/">Bill Freeman</a>, <a href="http://web.mit.edu/cocosci/josh.html">Joshua Tenenbaum</a>, <a href="http://syntheticneurobiology.org/people/display/71/11">Ed Boyden</a>, <a href="http://nancysbraintalks.mit.edu/">Nancy Kanwisher</a>, <a href="http://zlab.mit.edu/index.html">Feng Zhang</a>, and <a href="https://mcgovern.mit.edu/principal-investigators/tomaso-poggio">Tomaso Poggio</a>. Learned about how neuronal cells can be stretched for physical magnification and imaging, how CRISPR can bring us neurological therapeutics, and the specialized brain areas that we may have for visual words and music. But those are just the topics I will not talk about.<br />
Intrigued to learn more anyway?<br />
Here are some related articles:<br />
<a href="http://news.mit.edu/2014/crispr-technique-determines-gene-function-1210">http://news.mit.edu/2014/crispr-technique-determines-gene-function-1210</a><br />
<a href="http://news.mit.edu/2015/faculty-profile-edward-boyden-0522">http://news.mit.edu/2015/faculty-profile-edward-boyden-0522</a><br />
<a href="http://news.mit.edu/2015/neural-population-music-brain-1216">http://news.mit.edu/2015/neural-population-music-brain-1216</a><br />
<br />
In this post, I will focus instead on the common strands passing through the works of Bill Freeman, Joshua Tenenbaum, Josh McDermott, and Nancy Kanwisher - both to highlight the great interdisciplinary collaborations happening at MIT, and to give a broader sense of how neuroscience and computer science are informing each other, and leading to cool new insights and innovations.<br />
<br />
Bill Freeman presented the work spearheaded by his graduate student <a href="http://andrewowens.org/">Andrew Owens</a>: <a href="http://arxiv.org/abs/1512.08512">"Visually indicated sounds"</a>. Teaming up with Josh McDermott, who studies <a href="http://web.mit.edu/jhm/www/">computational audition</a> at the MIT Department of Brain and Cognitive Sciences, they linked sound to material properties and vice versa. Given a silent video as input (of a wooden stick hitting or scratching some surface), the team developed an algorithm that synthesizes realistic sound to go along with it. To do so they needed to convert videos of different scenes (with a mixture of materials) into some perceptually-meaningful space, and link them to sounds that were also represented in some perceptually-meaningful way. What does "perceptually-meaningful" refer to? The goal is to transform the complex mess that is colored pixels and audio waveforms into some stable representations that allows similar materials to be matched together and associated with the same material properties. For instance, pictures (and videos) of different foliage will look very different from each other (the shape and the color may have almost no pixel-overlap) and yet, somehow, the algorithm needs to discover the underlying material similarity.<br />
<br />
Here is one place where CNNs (convolutional neural nets) have been successful at transforming a set of pixels into some semantic representation (enough to perform scene recognition, object detection, and the other high-level computer vision tasks that the academic and industry communities have recently flooded the media outlets with). CNNs can learn almost human-like associations between images and semantics (like labels) or between images and other images. Owens and colleagues used CNNs to represent their silent video frames.<br />
<br />
On the sound side of things, waveforms were converted into "cochleagrams" - stable representations of sound that allow waveforms coming from similar sources (e.g. materials, objects) to be associated with each other even if individual timestamps of the waveforms have almost no overlap. Now to go from silent video frames to synthesized sounds, RNNs (recurrent neural nets) were used (RNNs are great for representing and learning sequences, by keeping around information from previous timesteps to make predictions for successive timesteps). The cochleagrams predicted by the RNNs could then be transformed back into sound, the final output of the algorithm. More details in <a href="http://arxiv.org/abs/1512.08512">their paper</a>.<br />
<br />
This work is a great example of the creative new problems that computer vision researchers are tackling. With the powerful representational architectures that deep neural networks provide, higher and higher-level tasks can be achieved - tasks that we would typically associate with human imagination and creativity (e.g. inferring what sound is emitted by some object, what lies beyond the video frame, what is likely to happen next, etc.). In turn, these powerful architectures are interesting from a cognitive science perspective as well: how are the artificial neural networks representing different features? images? inputs? What kinds of associations, correlations, and relationships do they learn from unstructured visual data? Do they learn to meaningfully associate semantically-related concepts? Cognitive scientists can give computer scientists some ideas about which representations may be reasonable for different tasks, given what is known from decades of experiments on the human brain. But the other side of the story is that computer scientists can prod these artificial networks to learn about the representational choices that the networks have converged on, and then cognitive scientists can design experiments to check if the networks in the human brain do the same. This allows the exploration of a wide space of hypotheses at low cost (no poking human brains required), to narrow down the focus of cognitive scientists in asking whether the human brain has converged on similar representations (or if not, how can it be more optimal?)<br />
<br />
Nancy Kanwisher mentioned how advances in deep neural networks are helping to understand functional representation in the brain. Kanwisher has done pioneering work on functional specialization in the brain (which brain areas are responsible for which of our capabilities) - including discovering the fusiform face area (FFA). In her talk, she discussed how the "Principle of Modular Design" (Marr, 1982) just makes sense - it is more efficient. She mentioned some examples of work from MIT showing there are <a href="http://nancysbraintalks.mit.edu/video/neuroanatomy-lesson-directors-cut">specialized areas</a> for faces, language, visual words, even theory of mind. By giving human participants different tasks to do and scanning their brain (using fMRI), neuroscientists can test hypotheses about the function of different brain regions (they check whether the brain signal in those regions changes significantly as they give participants different tasks to do). Some <a href="http://www.cell.com/current-biology/abstract/S0960-9822(12)01074-3?_returnURL=http%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0960982212010743%3Fshowall%3Dtrue">experiments</a>, for instance, have demonstrated that certain language-specific areas of the brain are not involved during logic tasks, arithmetic, or music (tasks that are sometimes hypothesized to depend on language). <a href="http://web.mit.edu/zsaygin/www/publications.html">Experiments</a> have shown that the brain's specialization is not all natural selection, and that specialized brain areas can develop as a child learns. Other <a href="http://web.mit.edu/bcs/nklab/media/SVNH_NGK_JMD_2015.pdf">experiments</a> (with Josh McDermott) have shown that uniquely-human brain regions exist, like ones selective to music and human speech (but not other sounds). Other experiments probe causality: what happens if specific brain regions are stimulated or dysfunctional? How are the respective functions affected or impaired? <a href="http://nancysbraintalks.mit.edu/video/what-happens-when-you-stimulate-face-area">Interestingly</a>, stimulating the FFA using electrodes can cause people's representations of faces to change. <a href="https://www.ted.com/talks/rebecca_saxe_how_brains_make_moral_judgments?language=en">Correspondingly</a>, stimulating other areas of the brain using TMS can cause moral judgements to shift.<br />
<br />
Kanwisher is now working with Josh Tenenbaum to look for areas of the brain that might be responsible for intuitive physical inference. Initial findings are showing that the regions activated during intuitive physics reasoning are the same ones responsible for action planning and motor control. Knowing how various functional areas are laid out in the brain, how they communicate with each other, and which resources they pool together, can help provide insights for new artificial neural architectures. <a href="http://web.mit.edu/yamins/www/publications.html">Conversely</a>, artificial neural architectures can help us support or cast doubt on neuroscience hypotheses by replicating human performance on tasks using different architectures (not just the ones hypothesized).<br />
<br />
Josh Tenenbaum is working on artificial architectures that can make the same inferences humans make, but also make the same mistakes (for instance, <a href="http://www.theinquirer.net/inquirer/feature/2434242/facebook-s-ai-tech-mimics-how-humans-learn">Facebook's AI</a> that reasons about the stability of towers of blocks, makes different incorrect predictions than humans). The best CNNs today are great at the tasks for which they are trained, sometimes even outperforming humans, but often also making very different mistakes. Why is it not enough to just get right what humans get right, without also having to get wrong what they get wrong? The mistakes humans make are often indicative of the types of broad inferences they are capable of, and uncover the generalizing power of the human mind. This is why one-shot learning is possible: humans can learn whole new concepts from a single example (and Tenenbaum has many <a href="http://videolectures.net/nips2010_tenenbaum_hgm/">demos</a> to prove it). This is why we can explain, imagine, problem solve, and plan. Tenenbaum says: "intelligence is not just pattern recognition. It is about modeling the world", and by this he means "analysis by synthesis".<br />
<br />
Tenenbaum wants to re-engineer "the game engine in your head". His group is working on probabilistic programs that can permit causal inference. For <a href="http://news.mit.edu/2015/better-probabilistic-programming-0413">example</a>, their algorithm can successfully recognize the parameters of a face (shape and texture; the layout and type of facial features) as well as the lighting and viewing angle used for the picture. Their algorithm does this by sampling from a generative model that iteratively creates and refines new faces, and then matching the result to the target face. Once a match is found, the parameters chosen for the synthesized face can be considered a good approximation for the parameters of the target face. Interestingly, this model, given similar tasks as humans (e.g. to determine if two faces are the same or different) takes similar amounts of time (corresponding to task difficulty) and makes similar mistakes. This is a good hint that the human brain might be engaged in a similar simulation/synthesis process during recognition tasks.<br />
<br />
Tenenbaum and colleagues have made great strides in showing how "analysis by synthesis" can be used to solve and achieve state-of-the-art performance on difficult tasks like face recognition, pose estimation, and character identification (<a href="http://news.mit.edu/2015/computer-system-passes-visual-turing-test-1210">even passing the visual Turing test</a>). As is the case for much of current neural network research, the original inspiration comes from the 80s and 90s. In particular, Hinton's <a href="https://en.wikipedia.org/wiki/Helmholtz_machine">Helmholtz Machine</a> had a wake-sleep cycle where recognition tasks were interspersed with a type of self-reinforcement (during "sleep") that helped the model learn on its own, even when not given new input. This approach helps the model gain representational power, and might give some clues as well about human intelligence (what do we do when we sleep?).<br />
<br />
How does the human mind make the inferences it does? How does it jump to its conclusions? How does it transfer knowledge gained on one task and apply it to a novel one? How does it learn abstract concepts? How does it learn from a single example? How does the human mind represent the world around it, and what physical structures are in place in order to accomplish this? How is the brain wired? These questions are driving all of the research described here and will continue to pull together the efforts of neuroscientists and computer scientists in the coming years more than ever before. Our new and ever-developing tools for constructing artificial systems and probing into natural ones are establishing more and more points of contact between fields. Symposia such as these can give one a small hint of what the tip of the iceberg might look like.Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com0tag:blogger.com,1999:blog-338911558031950356.post-11041407243858896902015-12-30T13:54:00.002-08:002015-12-30T14:05:02.483-08:00"Computer Behind Pixar": Teaching Computational Thinking for 3D Modeling and Representation<span style="color: #38761d;"><b>How do you teach a group of middle- or high-schoolers about computer graphics without setting them down in a computer lab or showing them code?</b></span> How do you teach them about 3D geometry without writing down a single mathematical formula on the board? And how, without doing all these things, can you nevertheless equip them with the vocabulary and intuition to be able to discuss and understand concepts in computer graphics, geometry, and representation?<br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGyTRaLb7qL3PNlO3gv33886nIa5LCBrMYRe5GYXwwYfTEmUTf92KJiDSSNb31sX8t2N2vSN0QzvcxFJwovIKJE4nWK1B_C53yVnTRm5jODhNSgMfo6F-csFQPhmrHDGdOLTxO8HM6OVJl/s1600/770801-MLB20397196410_082015-O.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGyTRaLb7qL3PNlO3gv33886nIa5LCBrMYRe5GYXwwYfTEmUTf92KJiDSSNb31sX8t2N2vSN0QzvcxFJwovIKJE4nWK1B_C53yVnTRm5jODhNSgMfo6F-csFQPhmrHDGdOLTxO8HM6OVJl/s320/770801-MLB20397196410_082015-O.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><span style="font-size: x-small;">Pinart toy: what better way to explain height fields?</span></td></tr>
</tbody></table>
<br />
<div style="font-style: normal; margin-bottom: 0in; orphans: 1;">
<span style="font-family: Times;">That was our
goal, and our chosen plan of attack was to flood the senses: let our
students touch and explore physical models, work through group
activities, watch video clips, participate in class discussions, and
see demos. <b><span style="color: #38761d;">We filled our classroom with 3D printed models of various
materials, faceted animal shapes, wooded mannequins, pin art boards,
crayons and fuzzy pipe cleaners.</span></b> </span></div>
<div style="font-style: normal; margin-bottom: 0in; orphans: 1;">
<span style="font-family: Times;">What do all these objects have in
common?</span></div>
<br />
<div style="margin-bottom: 0in; orphans: 1;">
<span style="font-family: Times;">These physical
models serve as examples and applications of different
representational choices, including voxel grids, meshes, and height
fields. Having <span style="color: #bf9000;"><b>physical examples</b></span> to point to and explore can launch a discussion of different representational (3D modeling) choices.</span><br />
<span style="font-family: Times;"><br /></span></div>
<div style="font-style: normal; margin-bottom: 0in; orphans: 1;">
<h3>
<span style="color: black;"><span style="font-family: Times;"><b>Splash
2015 @ MIT</b></span></span></h3>
</div>
<div style="margin-bottom: 0in; orphans: 1;">
<span style="color: black;"><span style="font-family: Times;"><span style="font-style: normal;"><span style="font-weight: normal;">On
November 22, 2015, <a href="http://people.csail.mit.edu/hishin/">Hijung Valentina Shin</a>, <a href="http://people.csail.mit.edu/aschulz/">Adriana Shulz</a>, and I taught
a 2-hour high-school class as part of </span></span></span></span><a href="https://esp.mit.edu/learn/Splash/index.html">MIT's
yearly Splash! program</a><span style="color: black;"><span style="font-family: Times;"><span style="font-style: normal;"><span style="font-weight: normal;"> -
a Fall weekend during which thousands of high-schoolers flood hundreds
of MIT's classrooms to be taught anything and everything. </span></span></span></span></div>
<div style="margin-bottom: 0in; orphans: 1;">
<br /></div>
<div style="font-style: normal; font-weight: normal; margin-bottom: 0in; orphans: 1;">
<span style="color: black;"><span style="font-family: Times;">In our Splash!
classroom, we sought to ask and answer the following questions: How
is an animated character created? How can we represent different
types of 3D structures? What kind of modeling decisions are made for
a special effects film? What techniques do anthropological
reconstruction, 3D printing, and game design have in common?</span></span></div>
<div style="font-style: normal; font-weight: normal; margin-bottom: 0in; orphans: 1;">
<span style="color: black;"><span style="font-family: Times;"><br /></span></span></div>
<div style="font-style: normal; font-weight: normal; margin-bottom: 0in; orphans: 1;">
<span style="font-family: Times;">Importantly, we
believed that these questions could be answered <span style="color: #bf9000;"><b>on an intuitive
level</b></span>, with no mathematical prerequisites. What better way to
motivate the study of the mathematical and computational sciences
than to give students a faint whiff of the awesome things they would
be able to accomplish and think about in greater depth if armed with
the right tools? </span><br />
<span style="font-family: Times;"><br /></span>
<span style="color: #38761d; font-family: Times;"><b>Computational thinking to the rescue!</b></span></div>
<div style="font-style: normal; font-weight: normal; margin-bottom: 0in; orphans: 1;">
<br /></div>
<div style="font-style: normal; font-weight: normal; margin-bottom: 0in; orphans: 1;">
<span style="color: black;"><span style="font-family: Times;">Here I will
briefly outline the structure of our 2-hour class and the decisions made along the way, to provide possible inspiration for
similar classroom activities and lessons. For the benefit of others, we have made all our slides <a href="http://web.mit.edu/zoya/www/representation_in_graphics.pdf">available online</a>.</span></span><br />
<span style="color: black;"><span style="font-family: Times;"><br /></span></span></div>
<div style="font-style: normal; margin-bottom: 0in; orphans: 1;">
<h3>
<span style="color: black;"><span style="font-family: Times;"><b>Coding
without coding</b></span></span></h3>
</div>
<div style="font-style: normal; margin-bottom: 0in; orphans: 1;">
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgV9q8InPZbhbThpZXTB71xEVUITwILgDe9Jzn9a-pF19XFcv4SkOYWLu6lQHz_iKzAfiwTHNUo4VNcAoI1ZDGmz07MWli6lWbQIEarXbBQ7nj3V02k5CC97paxhBG6dLeg9-CYahMet0Wz/s1600/Screen+Shot+2015-12-29+at+12.50.27+PM.png" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto; text-align: center;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgV9q8InPZbhbThpZXTB71xEVUITwILgDe9Jzn9a-pF19XFcv4SkOYWLu6lQHz_iKzAfiwTHNUo4VNcAoI1ZDGmz07MWli6lWbQIEarXbBQ7nj3V02k5CC97paxhBG6dLeg9-CYahMet0Wz/s320/Screen+Shot+2015-12-29+at+12.50.27+PM.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Target shape that one student described to the other using only<br />
a set of provided primitives: colored squares, line segments, or<br />
polygonal shapes.</td></tr>
</tbody></table>
Our <span style="color: #bf9000;"><b>ice-breaker
activity</b></span> first introduced the concepts of <b><span style="color: #38761d;">representational primitives</span></b>
and algorithmic decisions. Students split up into pairs, armed with grids and sketching utensils (colored crayons or pencils). One student
was given a target shape, a set of primitives, and instructions.
The goal was to supply one's partner with a sufficient and clear
recipe to reproduce the target shape as accurately as possible. Some
students could only specify one grid cell at a time with coordinates and a target color. Another set of instructions armed
students with a ruler and the ability to specify starting and ending
coordinates of line segments. A third group of students had <a href="http://www.amazon.com/gp/product/B0044A41FC?psc=1&redirect=true&ref_=oh_aui_detailpage_o00_s00">polygonal shape rulers</a> – e.g. triangles, squares, circles. Students could tell their partners to center a shape at specific coordinates.<br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyvqB9-YACQ7vt6DUvGPYnl_IUCXcjYo0RSm70SsLBMyBcwB7GvLS8Qu8NWvnwevabbIQVufTYQLXx50lk2D33dywjYlORN_72krqT6LvkyG8ismeEf33kGz2YaGBhyphenhyphenKoV0fF6RmfxUEk0/s1600/Screen+Shot+2015-12-29+at+12.53.33+PM.png" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto; text-align: center;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyvqB9-YACQ7vt6DUvGPYnl_IUCXcjYo0RSm70SsLBMyBcwB7GvLS8Qu8NWvnwevabbIQVufTYQLXx50lk2D33dywjYlORN_72krqT6LvkyG8ismeEf33kGz2YaGBhyphenhyphenKoV0fF6RmfxUEk0/s200/Screen+Shot+2015-12-29+at+12.53.33+PM.png" width="121" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Polygonal primitives<br />
(<a href="http://www.amazon.com/gp/product/B0044A41FC?psc=1&redirect=true&ref_=oh_aui_detailpage_o00_s00">ordered on Amazon</a>)</td></tr>
</tbody></table>
<br />
Overall, we gave different student pairs different primitives:<br />
<ul>
<li>pixels (colored squares)</li>
<li>line segments</li>
<li>polygonal shapes</li>
</ul>
</div>
<div style="font-style: normal; font-weight: normal; margin-bottom: 0in; orphans: 1;">
</div>
<div style="font-style: normal; font-weight: normal; margin-bottom: 0in; orphans: 1;">
<span style="color: black;"><span style="font-family: Times;">We gave all
students the same amount of time to complete this activity in pairs
(15 minutes), after which students showed off their creations to
their partners and other students in the class. These creations
were hung around the classroom at the amusement of the students. </span></span>
</div>
<div style="font-style: normal; font-weight: normal; margin-bottom: 0in; orphans: 1;">
<br /></div>
<div style="font-style: normal; font-weight: normal; margin-bottom: 0in; orphans: 1;">
<span style="font-family: Times;">This gave us a
great launching pad for <span style="color: #bf9000;"><b>discussion</b></span> about the trade-offs between
representational accuracy and algorithmic efficiency. We asked
students: What did you find easy and hard? Were there parts of the
shape that were well represented by your primitives? Could everything
be represented by the primitives? What took you the longest? How many
individual primitives did you end up using?</span></div>
<div style="font-style: normal; font-weight: normal; margin-bottom: 0in; orphans: 1;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<div style="margin-bottom: 0in; orphans: 1;">
<span style="font-family: Times;">This kind of
activity (or variants of it) is a good <span style="color: #bf9000;"><b>intro to programming activity</b></span>, as students have to think about formalizing clear step-by-step instructions for their partner to carry out. The full instructions and templates for our activity are <a href="http://web.mit.edu/zoya/www/representation_in_graphics.pdf" style="color: black;">included here</a>.</span><br />
<span style="font-family: Times;"><br /></span>
<br />
<h3>
Computer behind Pixar</h3>
<span style="text-align: center;">Inspired by the recent hype around Pixar* and particularly <a href="http://www.mos.org/">Boston Museum of Science</a>'s temporary Pixar exhibit, we called our class "Computer behind Pixar". </span><span style="text-align: center;">The common goal of the exhibit and other educational media about Pixar is to hook in the general public with the beloved animations and characters for the purpose of introducing and motivating the underlying mathematical and scientific concepts. </span><span style="text-align: center;">In fact, Mike from Monsters Inc. served as a repeating element throughout our activities, though we branched beyond Pixar, and beyond animation more generally. </span><br />
<span style="text-align: center;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2VjxYGHP11aH_QyCXQFXoHKldkL2SuQF4NFxCQ5rIvt2efIZja-jaX9mVamXOTUwTdChHtTf_gzN-88G3NE9njbau4tbwalTEq2YnPFSGg2EZQl4MRhYD_tdOuBTb5o3OqVP42aVe3B5b/s1600/Screen+Shot+2015-12-29+at+2.28.40+PM.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2VjxYGHP11aH_QyCXQFXoHKldkL2SuQF4NFxCQ5rIvt2efIZja-jaX9mVamXOTUwTdChHtTf_gzN-88G3NE9njbau4tbwalTEq2YnPFSGg2EZQl4MRhYD_tdOuBTb5o3OqVP42aVe3B5b/s320/Screen+Shot+2015-12-29+at+2.28.40+PM.png" width="320" /></a></div>
<span style="text-align: center;"><br /></span>
<span style="text-align: center;"><br /></span>
<br />
<div style="text-align: left;">
<span style="text-align: center;"><span style="font-size: x-small;">* Reference links on the topic of math behind Pixar:</span></span></div>
<div style="text-align: left;">
<span style="text-align: center;"><a href="https://www.khanacademy.org/partner-content/pixar"><span style="font-size: x-small;">Khan Academy's Pixar in a Box</span></a></span></div>
<div style="text-align: left;">
<a href="http://ed.ted.com/lessons/pixar-the-math-behind-the-movies-tony-derose"><span style="font-size: x-small;">Ted talk about math behind Pixar</span></a></div>
<div style="text-align: left;">
<a href="https://www.youtube.com/watch?v=3Iu1Z0h1i1Y"><span style="font-size: x-small;">Intro to Boston Museum of Science's Pixar exhibit</span></a></div>
<div style="text-align: left;">
<span style="font-size: x-small;"><a href="http://www.scientificcomputing.com/news/2015/07/building-computational-thinkers-exhibit-offers-unique-look-science-behind-pixar">Article about the science behind Pixar</a></span></div>
<div style="text-align: left;">
<a href="http://www.businessinsider.my/inside-out-review-2015-5/#dSgQ1fYR4uJX0ZmJ.97"><span style="font-size: x-small;">Business Insider article about "Inside Out"</span></a></div>
<div style="text-align: left;">
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
</div>
<div class="separator" style="clear: both; text-align: left;">
We described and <a href="https://youtu.be/3Iu1Z0h1i1Y?t=47">showed a video</a> about the rendering pipeline*, and drew attention to the importance of modeling at the core of this pipeline, as the initial step that all future steps crucially depend on. We defined <span style="color: #38761d;"><b>modeling as a mathematical representation composed of primitives</b></span>. </div>
<div class="separator" style="clear: both; text-align: left;">
The rest of our discussion centered around different representational choices and their properties.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFfSG5qibE5_To9csiJQsUxIZwoSmFz-hJcPIwDFOWFlhG-o6mQkvNvmUORVLORv7-1_UiIk3MSHrzOd6pFurk4Szu7ktMsTmZmPMp8Stn6EQBH-idA8PQryMz5TG81HeiAU9Z5b_230PJ/s1600/Screen+Shot+2015-12-30+at+11.20.19+AM.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="252" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFfSG5qibE5_To9csiJQsUxIZwoSmFz-hJcPIwDFOWFlhG-o6mQkvNvmUORVLORv7-1_UiIk3MSHrzOd6pFurk4Szu7ktMsTmZmPMp8Stn6EQBH-idA8PQryMz5TG81HeiAU9Z5b_230PJ/s320/Screen+Shot+2015-12-30+at+11.20.19+AM.png" width="320" /></a></div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2VjxYGHP11aH_QyCXQFXoHKldkL2SuQF4NFxCQ5rIvt2efIZja-jaX9mVamXOTUwTdChHtTf_gzN-88G3NE9njbau4tbwalTEq2YnPFSGg2EZQl4MRhYD_tdOuBTb5o3OqVP42aVe3B5b/s1600/Screen+Shot+2015-12-29+at+2.28.40+PM.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><br class="Apple-interchange-newline" /></a>
<span style="font-size: x-small;"><br /></span>
<span style="font-size: x-small;"><br /></span>
<span style="font-size: x-small;"><br /></span>
<span style="font-size: x-small;">* More rendering resources:</span><br />
<span style="font-size: x-small;"><a href="https://www.youtube.com/watch?v=3Iu1Z0h1i1Y&feature=youtu.be&t=47">Video about rendering in Pixar</a></span><br />
<span style="font-size: x-small;"><a href="https://www.fxguide.com/featured/inside-out-rendering/">Article about rendering in "Inside Out"</a></span><br />
<span style="font-size: x-small;"><a href="http://www.treddi.com/app/en/articoli/pagina/924-the-making-of-the-dark-knight">Character rendering (dark knight)</a></span><br />
<span style="font-size: x-small;"><a href="https://ev111426.wordpress.com/2014/09/26/5/">Rendering pipeline summary</a></span><br />
<h3>
</h3>
<h3>
</h3>
<h3>
</h3>
<div>
<br /></div>
<h3>
</h3>
<h3>
</h3>
<h3>
</h3>
<h3>
</h3>
<h3>
</h3>
<div>
<br /></div>
<h3>
</h3>
<h3>
</h3>
<h3>
</h3>
<h3>
</h3>
<h3>
</h3>
<h3>
</h3>
<h3>
</h3>
<h3>
</h3>
<h3>
</h3>
<div>
<br /></div>
<h3>
Tangible examples of 3D representations</h3>
<div>
<br /></div>
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiwQYzkFrzDv2jzj1_E8_U9tfwnRW8vzQUdNFOGJRBfXhOs-UT_ZDEha0CvONxxbVP6t_nlWcJr7U800nxel9mU4MbEHft8oxhVzhjFARenv1w4m0Q7z9209GVbB4QVk_OEzKEDI1LXa-3N/s1600/Screen+Shot+2015-12-30+at+11.42.31+AM.png" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="151" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiwQYzkFrzDv2jzj1_E8_U9tfwnRW8vzQUdNFOGJRBfXhOs-UT_ZDEha0CvONxxbVP6t_nlWcJr7U800nxel9mU4MbEHft8oxhVzhjFARenv1w4m0Q7z9209GVbB4QVk_OEzKEDI1LXa-3N/s200/Screen+Shot+2015-12-30+at+11.42.31+AM.png" width="200" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">3D printed models are a tangible<br />
demonstration of discretization and<br />
the resolution issue.</td></tr>
</tbody></table>
<h4>
</h4>
<h4>
</h4>
<h4>
Voxel grids</h4>
<div>
We introduced the concept of <span style="color: #38761d;"><b>discretization</b></span>, necessary for the representation of shapes in digital computers: 2D shapes as <span style="color: #38761d;"><b>pixels</b></span> and 3D shapes as <span style="color: #38761d;"><b>voxels</b></span>. We reminded students of the ice-breaker activity where grid cells were used as primitives. </div>
<div>
We then discussed <span style="color: #38761d;"><b>voxel grids</b></span> as one form of representation for 3D objects, commonly used for 3D printing. We talked about the <span style="color: #38761d;"><b>resolution issue</b></span>: the trade-off between accuracy and efficiency. We passed around <span style="color: #bf9000;"><b>physical 3D printed models</b></span> at various resolutions, similar to the models pictures on the right.</div>
<div>
<br /></div>
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVUQ0DKQf5zCMCph0ANjwowVAwHR6te1oC0A4oDivSPIZ7UfIf5rvTcxW4-RPKBenlMlU5fx1j0rb_mh8JfqLendLadT_VsPY6AVWPQehQlDAFPa1z4mHvbOAEMwqAh7DoAYmxZcQ-VZLN/s1600/IMG_20151217_131447464.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="137" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVUQ0DKQf5zCMCph0ANjwowVAwHR6te1oC0A4oDivSPIZ7UfIf5rvTcxW4-RPKBenlMlU5fx1j0rb_mh8JfqLendLadT_VsPY6AVWPQehQlDAFPa1z4mHvbOAEMwqAh7DoAYmxZcQ-VZLN/s200/IMG_20151217_131447464.jpg" width="200" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Physical models to demonstrate the<br />
differences between volumetric and<br />
boundary representations. One is much<br />
lighter! Why? It requires less material<br />
to represent (and store).</td></tr>
</tbody></table>
<h4>
Triangular meshes</h4>
<div>
In talking about efficiency, we introduced the notion of <span style="color: #38761d;"><b>boundary representations</b></span>, specifically <b><span style="color: #38761d;">meshes</span></b>, for representing 3D objects without having to represent and explicitly store all the internal voxels (the volume). </div>
<div>
We connected the boundary representation to the ice-breaker activity, where in 2D, line segments were used to represent the target shape's boundary. We then showed students a <b style="color: #bf9000;">demo of MeshLab</b>, and passed around physical examples of volumetric and boundary representations.<br />
<br /></div>
<h4>
CSG</h4>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgic25l1EJLptDaXCfw2SPNla9G4Ma5vJVTRebw-AL576Yv0nXEu9mYv074jgKT2ZV7thoqJgVvrGeVoZDrBBZQRQ-XTZBTtKc05aRh8RJuKgH1eM-unAa1d_wxrEb8mBN82LKaESaRHmJK/s1600/Screen+Shot+2015-12-30+at+12.23.19+PM.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgic25l1EJLptDaXCfw2SPNla9G4Ma5vJVTRebw-AL576Yv0nXEu9mYv074jgKT2ZV7thoqJgVvrGeVoZDrBBZQRQ-XTZBTtKc05aRh8RJuKgH1eM-unAa1d_wxrEb8mBN82LKaESaRHmJK/s1600/Screen+Shot+2015-12-30+at+12.23.19+PM.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZESWnPhTc4-1YgdRhsJkKD1C4zJ68l8FzT-RuAPUn5WjDBUIhT3pAxs5CXmdMXEGpjG2nzSV49Wr0usItKL9Me6Wm4SgJaOODcDHpXmkTz0h1kA8f15fh2AfUcIheop2TnyscrBRtwO5d/s1600/Screen+Shot+2015-12-30+at+12.23.19+PM.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><br /></a>We moved on to discuss how simple shapes can be combined with different operations to create more complex shapes, in 3D via <b><span style="color: #38761d;">constructive solid geometry (CSG)</span></b>. We reminded students that the ice-breaker activity also contained polygonal primitives in 2D. For 3D, we showed students a <span style="color: #bf9000; font-weight: bold;">demo of OpenScad</span> and discussed primitive operations (union, intersection, difference, ...) that can be performed on shapes. Applications in manufacturing were discussed. </div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<h4 style="clear: both; text-align: left;">
Height Fields</h4>
<div class="" style="clear: both; text-align: left;">
<span style="color: #38761d;"><b>Heigh fields</b></span> were introduced with the help of <a href="http://www.amazon.com/Adorox-Plastic-Novelty-Multiple-Colors/dp/B00P6S4W0U/ref=sr_1_18?s=toys-and-games&ie=UTF8&qid=1446837415&sr=1-18&keywords=pin+art">pin art boards</a>, as pictured at the beginning of this article. Students played with the pin boards and considered again the concepts of discretization and the representation issue. We asked students: which kind of shapes or surfaces can be represented this way and which can not?<br />
<br /></div>
<h4 style="clear: both; text-align: left;">
Procedural Modeling</h4>
<div class="separator" style="clear: both; text-align: left;">
</div>
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3pQbcSfSVpzdeFCF770d8TvWkjGzal_6K_geg1UNBCoK_23pNIg_vD6pY4UZLAfHamf05sD27FocYBp4_0qloYFDv4klIBGDUFt1-RQfJge2E2j3UujTtTL2HTjlKmsroKEOgWdq-yMd2/s1600/grassbrave.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3pQbcSfSVpzdeFCF770d8TvWkjGzal_6K_geg1UNBCoK_23pNIg_vD6pY4UZLAfHamf05sD27FocYBp4_0qloYFDv4klIBGDUFt1-RQfJge2E2j3UujTtTL2HTjlKmsroKEOgWdq-yMd2/s320/grassbrave.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The grass in Pixar's Brave was created with procedural modeling,<br />
using parametric curves and randomness. <br />
A great hands-on demo of this kind of modeling can be found on:<br />
<a href="https://www.khanacademy.org/partner-content/pixar/environment-modeling-2/animating-parabolas-ver2/a/start-here-em">Khan Academy's Pixar-in-a-Box</a>.</td></tr>
</tbody></table>
We discussed how shapes could be created by specifying <span style="color: #38761d;"><b>procedures on primitives</b></span> (aside from the primitive operations in CSG). We showed demos of <span style="color: #38761d;"><b>solids of revolution</b></span> (<i>what better way to motivate the concept that for most students appears for the first time only in college calculus?</i>). We discussed how procedures like <span style="color: #38761d;"><b>revolution</b></span> and <span style="color: #38761d;"><b>extrusion</b></span> can be performed along different paths to create all sorts of complex shapes. We discussed how these paths can be further <b><span style="color: #38761d;">parametrized</span></b> so that the revolution or extrusion procedure changes along the path. We introduced <span style="color: #38761d;"><b>randomness</b></span> as another concept that can be used to add variability to the representation.<br />
<div class="separator" style="clear: both; text-align: left;">
We discussed applications to modeling trees, forests, grassy fields, crowds, and cities.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<u>3D Representation Primitives Operations (recipe) </u></div>
<div class="separator" style="clear: both; text-align: left;">
Voxel grids Voxels Material specification for each voxel</div>
<div class="separator" style="clear: both; text-align: left;">
Triangle mesh Triangles List of triangles with locations</div>
<div class="separator" style="clear: both; text-align: left;">
CSG Basic shapes CSG operations (union, intersection, etc.)</div>
<div class="separator" style="clear: both; text-align: left;">
Height field Points with height Assignment of heights to points</div>
<div class="separator" style="clear: both; text-align: left;">
Procedural model Basic shapes Procedure (e.g. extrusion along path)</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<h3>
</h3>
<h3>
</h3>
<h3>
A new way to look at things</h3>
<div class="" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh7FjayDjm0xNmEMgb8uMhJ9mmQ-0JrDwaLgl1eFGZHu2KjS6bn0OI40Qx2JFJvSM-VXEmoT83tWi7HaAz0jHDG3VBIt2r0nB6ULTP7q6rcFPQcgt-AB8LcK26ue5FQ8x2bubZ8XWvst0N1/s1600/Screen+Shot+2015-12-30+at+1.17.19+PM.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="149" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh7FjayDjm0xNmEMgb8uMhJ9mmQ-0JrDwaLgl1eFGZHu2KjS6bn0OI40Qx2JFJvSM-VXEmoT83tWi7HaAz0jHDG3VBIt2r0nB6ULTP7q6rcFPQcgt-AB8LcK26ue5FQ8x2bubZ8XWvst0N1/s200/Screen+Shot+2015-12-30+at+1.17.19+PM.png" width="200" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKrU3OoW2waS5huoH3MlW4_S4p-aVA8MKBqm_jnBISl7I5eKmC_ymBSzgjyD6ZBxDrRq7l6nXWGBRaV0NFjntaBviG5N2G0O7tUr1qswapKkIXWRMkyHh25_nGEjpdiknRwDOmhyJlQvWX/s1600/Screen+Shot+2015-12-30+at+1.16.35+PM.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKrU3OoW2waS5huoH3MlW4_S4p-aVA8MKBqm_jnBISl7I5eKmC_ymBSzgjyD6ZBxDrRq7l6nXWGBRaV0NFjntaBviG5N2G0O7tUr1qswapKkIXWRMkyHh25_nGEjpdiknRwDOmhyJlQvWX/s200/Screen+Shot+2015-12-30+at+1.16.35+PM.png" width="166" /></a>With our class, we hoped to give students a look at the modeling decisions that underly all the animated films, video games, and special effects they see on a daily basis. We wrapped up our class with a <span style="color: #bf9000; font-weight: bold;">thought exercise</span>, putting students in the position of making decisions about how to model different objects. We told them to think about the different representations we discussed: the <span style="color: #38761d;"><b>primitives and operations</b></span> required. We told them to consider the <span style="color: #38761d;"><b>trade-off between accuracy and efficiency</b></span>. Given a representation, we also told them to think about its <b><span style="color: #38761d;">usability</span></b> - what kind of use cases are being considered, e.g. whether the modeled object needs to be animated and how. Students were asked to brainstorm how they would model the following objects: buildings, cities, fabric, hair, grass, water. Along the way, we showed them <span style="color: #bf9000;"><b>image and video demos </b></span>(all these links can be found <a href="http://web.mit.edu/zoya/www/representation_in_graphics.pdf">in the slides</a>). We passed around more <span style="color: #bf9000;"><b>physical models</b></span>. Together, we watched <a href="https://www.youtube.com/watch?v=MnQLjZSX7xM">a video "behind special effects"</a> that showcased the kinds of 3D models used in movies, a great visual review of the many representations covered in our class. We told students to look around and realize that 3D modeling decisions underlie many other applications: special effects in films, video games, simulations, anthropological reconstructions, product design, urban planning, robotics, and 3D printing. To be reminded that they have been armed with a new way to look at things, students took home <a href="http://www.redbubble.com/shop/polygon+stickers">polygonal stickers</a>.</div>
<div class="" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmYNBGDqhyphenhyphenrJWvxd3jPCpJQtFYG3iY0kIVOMIqQEMRto2lr-i33dqTOTeRSB-AFRR70j8X9PGzO1sxs1MrLCr_IkYPQ2J4cmIBA5tNzNYpZdzuWVEW3TkHzTITsykWJ9O0kfM0MDmgH3si/s1600/Screen+Shot+2015-12-30+at+1.31.41+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="184" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmYNBGDqhyphenhyphenrJWvxd3jPCpJQtFYG3iY0kIVOMIqQEMRto2lr-i33dqTOTeRSB-AFRR70j8X9PGzO1sxs1MrLCr_IkYPQ2J4cmIBA5tNzNYpZdzuWVEW3TkHzTITsykWJ9O0kfM0MDmgH3si/s200/Screen+Shot+2015-12-30+at+1.31.41+PM.png" width="200" /></a></div>
<br />
<br />Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com0tag:blogger.com,1999:blog-338911558031950356.post-42796811609668660522015-08-24T21:47:00.001-07:002015-08-24T21:47:56.870-07:00Hyperconnectedness leads to hyperactivityAlthough the term "hyperconnected" <a href="http://www.oxforddictionaries.com/us/definition/american_english/hyperconnected">already exists</a>, I will use the following user-centric definition: the embedding of an individual within the internet - the individual's omnipresence on the web. People have all sorts of outlets for posting, storing, and sharing all sorts of content: for example, you can post your photos on Facebook, Google Photos, Instagram, Flikr, Snapchat, etc.; you can blog on Blogger, Wordpress, Tumblr, etc.; you can write about articles, news, your day and your thoughts on Twitter, Facebook, Google+, etc.; you can exchange information on Quora, Reddit, etc. and links on Pintrest and Delicious; you can share your professional information on LinkedIn, your video creations on YouTube, Vimeo, and Vine. You get the point. Although there is some redundancy to some of these internet services, they also have enough of their own features that they can be tailored for particular use cases (not to mention slightly different communities and audiences). I've personally found that there are enough differentiating features (at this point at least) to warrant separate posts on separate sites. And what does this all lead to? Hyperactivity, I claim.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://d1lwft0f0qzya1.cloudfront.net/dims4/COKE/e5c0bcf/2147483647/thumbnail/596x334/quality/75/?url=http%3A%2F%2Fassets.coca-colacompany.com%2Fbd%2Fae%2Fe9124ad246a09a3e6bada599f629%2F5-tools-for-staying-tech-savvy-604.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://d1lwft0f0qzya1.cloudfront.net/dims4/COKE/e5c0bcf/2147483647/thumbnail/596x334/quality/75/?url=http%3A%2F%2Fassets.coca-colacompany.com%2Fbd%2Fae%2Fe9124ad246a09a3e6bada599f629%2F5-tools-for-staying-tech-savvy-604.jpg" height="179" width="320" /></a></div>
<div style="text-align: center;">
<span style="font-size: xx-small;">source: http://www.coca-colacompany.com/stories/5-tools-for-staying-tech-savvy-in-a-hyper-connected-world</span></div>
<br />
With so many ideas, thoughts, suggestions, and opinions swirling around, a whole world of possibilities opens up to the individual - from digesting all the content that is posted, to posting one's own content. The posts of others inspire one to create and do, and the social interconnectedness - the awareness that your content will be widely seen - drive one to post as well. This self-reinforcing vicious cycle is the perfect breeding ground for creativity and content-creation. We live not just in the information age - we live in the creativity age*. Yes, people have always created before, but now that creations are visible to the whole world, they can stand on the shoulders of creative giants. Ideas are exchanged and evolve at the speed of fiber optics. People hyperactively create.<br />
<br />
* side note: because creativity correlates with content-creation here, we're generating significantly more data than ever before; stay tuned for a very intelligent (and creative) Internet of Things!<br />
<br />
At this point, the discussion portion of this blog post ends, and I share my excitement for some of the awesomeness on the creative web below. These are the reasons why there are never enough hours in the day, or years in a lifetime. I am constantly inspired by how many different things people master and how creative they can be in all forms of things and activities. The rest of this post can be summarized as #peopleareawesome.<br />
<br />
The activities I list below may at first glance seem like a random sampling, but they're a set of activities that are united by the ability to do them without being a total expert (with some practice you can already achieve something!) and the ability to do them on the side (as a hobby, for short periods of time, with limited equipment).<br />
<br />
<u>Electronics, robotics, RC</u><br />
<u><br /></u>
My 15-year-old brother has learned to put together electronics and build RC vehicles, planes, boats, and drones by watching lots of YouTube videos. This is the type of knowledge that no traditional education can deliver at such a density and speed. This creative maker culture is largely driven by being part of a large community of like-minded individuals (no matter what age) that positively reinforce each other via posts, discussions, and likes. An individual not connected to the internet might have a very small community (if any) with much sparser positive reinforcement, which I claim would result in fewer amazing creations.<br />
<br />
<div style="text-align: center;">
more creations: <a href="http://rcgatorr.blogspot.ca/">http://rcgatorr.blogspot.ca/</a></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/Wz3eV1NA0ns/0.jpg" src="https://www.youtube.com/embed/Wz3eV1NA0ns?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<br />
<br />
<u>New art styles</u><br />
<u><br /></u>
Art is a hobby of mine, and I'm big on constantly trying new things. There's always something new that the web coughs up in this regard, beyond the traditional styles of sketching and painting. For instance, consider these widely diverging artistic styles:<br />
<br />
check out the art of <b>wood burning</b><br />
<div style="text-align: center;">
<a href="http://www.pyrographyonline.com/free_pyro_pattern.html">http://www.pyrographyonline.com/free_pyro_pattern.html</a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.pyrographyonline.com/files/temp/978_1_56523_482_6_567796126.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://www.pyrographyonline.com/files/temp/978_1_56523_482_6_567796126.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
and the art of <b>painting on birch bark</b></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://viola.bz/paintings-on-bark-by-sergey-surin/">http://viola.bz/paintings-on-bark-by-sergey-surin/</a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://viola.bz/wp-content/uploads/2012/12/artist-Sergey-Surin-7.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://viola.bz/wp-content/uploads/2012/12/artist-Sergey-Surin-7.jpg" height="256" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
and painting by <b>wet felting</b></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://livingfelt.wordpress.com/tag/felted-landscapes/">https://livingfelt.wordpress.com/tag/felted-landscapes/</a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://livingfelt.files.wordpress.com/2015/02/11-fly-me-to-the-moon.jpg?w=1494" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://livingfelt.files.wordpress.com/2015/02/11-fly-me-to-the-moon.jpg?w=1494" width="311" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
and check out this crazy video of <b>candle carving</b></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/5w9Dv6GT0ac/0.jpg" src="https://www.youtube.com/embed/5w9Dv6GT0ac?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
also check out: </div>
<div class="separator" style="clear: both; text-align: center;">
<b><a href="https://www.youtube.com/watch?v=VoJnfIScTOs">sand sculptures</a> </b>and<b> </b><b><a href="http://www.noupe.com/inspiration/showcases/40-insane-ice-sculptures.html">ice sculptures</a></b></div>
<br />
<u>Scrapbooking and crafting</u><br />
<u><br /></u>
<div style="text-align: center;">
personal memories and trips can be creatively captured in <b>scrapbooks</b></div>
<div style="text-align: center;">
in both physical form: <a href="http://diyready.com/cool-scrapbook-ideas-you-should-make/">http://diyready.com/cool-scrapbook-ideas-you-should-make/</a></div>
<div style="text-align: center;">
and via virtual tools: <a href="https://www.shutterfly.com/photo-books">https://www.shutterfly.com/photo-books</a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2v5n3a6rmc32nu8zf3mhydc1.wpengine.netdna-cdn.com/wp-content/uploads/2015/06/Cool-Scrapbook-Ideas-You-Should-Make-Travel-Scrapbook.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2v5n3a6rmc32nu8zf3mhydc1.wpengine.netdna-cdn.com/wp-content/uploads/2015/06/Cool-Scrapbook-Ideas-You-Should-Make-Travel-Scrapbook.jpg" height="400" width="142" /></a></div>
<u><br /></u>
<u>Culinary masterpieces</u><br />
<u><br /></u>
Judging by the popularity of cooking channels, and food-, cooking-, and baking-related tags and posts on different social networks, people love to share the dishes, recipes, and culinary masterpieces that they create. I mean, just look at this:<br />
<br />
<div style="text-align: center;">
<a href="https://www.pinterest.com/juanice/awesome-decorated-cakes/">https://www.pinterest.com/juanice/awesome-decorated-cakes/</a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://s-media-cache-ak0.pinimg.com/736x/51/f3/d1/51f3d1dfd7e52f2fb12e6d24f97e39b8.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://s-media-cache-ak0.pinimg.com/736x/51/f3/d1/51f3d1dfd7e52f2fb12e6d24f97e39b8.jpg" width="212" /></a></div>
<br />
<div style="text-align: center;">
and themed foods for any occasion: </div>
<div style="text-align: center;">
<a href="https://www.pinterest.com/robinsweb/christmas-party-foods/">https://www.pinterest.com/robinsweb/christmas-party-foods/</a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://s-media-cache-ak0.pinimg.com/736x/3d/37/13/3d3713cd84ec2f115ec95906677118b3.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://s-media-cache-ak0.pinimg.com/736x/3d/37/13/3d3713cd84ec2f115ec95906677118b3.jpg" width="86" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<u>Travel blogs and photography</u></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: left;">
I'm also hugely inspired by all the travel blogs people put together. Not only do they find the time to visit amazing places and capture them from all sorts of beautiful angles, they also blog about it: <a href="http://fathomaway.com/slideshow/fathom-2015-best-travel-blogs-and-websites1/">http://fathomaway.com/slideshow/fathom-2015-best-travel-blogs-and-websites1/</a></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The really creative also put together annotated, narrated, and musical slideshows and videos.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
I'm not even going to go into all the amazing photography people do. I will leave you with this:</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: center;">
<a href="http://www.beforethey.com/">http://www.beforethey.com/</a></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/HaZyp0TSqWM/0.jpg" src="https://www.youtube.com/embed/HaZyp0TSqWM?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<u>Data visualization</u></div>
<div style="text-align: left;">
<u><br /></u></div>
<div style="text-align: left;">
Both an art and a science, how to visually depict data is very relevant in this day and age. I'm inspired by creativity, once again:</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: center;">
<a href="http://flowingdata.com/2014/12/19/the-best-data-visualization-projects-of-2014-2/">http://flowingdata.com/2014/12/19/the-best-data-visualization-projects-of-2014-2/</a> </div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://i1.wp.com/flowingdata.com/wp-content/uploads/2014/12/Favorites-of-20141.png?zoom=2&fit=700%2C9999" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://i1.wp.com/flowingdata.com/wp-content/uploads/2014/12/Favorites-of-20141.png?zoom=2&fit=700%2C9999" height="121" width="320" /></a></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: left;">
<u>Creative writing</u></div>
<div style="text-align: left;">
<u><br /></u></div>
<div style="text-align: left;">
Other than blog writing, I like the idea of creating writing on-the-side to de-stress and get some brain juices flowing - here's some things worth trying and checking out (and possibly submitting to if you're extra adventurous): <a href="http://www.nature.com/nature/focus/arts/futures/">short SF stories</a>, poetry, <a href="http://contest.newyorker.com/CaptionContest.aspx">funny captions</a>.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Another form of "creative writing" is putting together <a href="http://web.mit.edu/zoya/www/docs.html">tutorials</a>, explanations, etc. on all sorts of topics that interest you. It allows you to organize your thoughts and attempt to explain some content with a specific audience in mind. I love to write, explain, and write explanations, but if only there was more time in a day...</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<u>How it all ties together</u></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
People inspire others by taking photos of their creations and posting them on photo-sharing sites, they create videos of the how-to process to motivate others to try, and they bookmark ideas/links they like. They then blog or tweet or chirp about their process and final products and otherwise share their creations with their social networks and the world. The resulting online interactions (sharing of ideas, discussions, comments, and likes) sparks the next cycle of creativity, and on it goes. (I posted some of the pictures above with the intention of inspiring others to try some new things as well.)</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
In short, there is no shortage of activities to occupy oneself with if there is some time on the side. Of all the activities and links listed above, I've tried about 70%. I am definitely hyperactive when it comes to creating, and the internet age is fueling a lot of that for me by constantly feeding me new ideas. I believe that when you try new things, you expand your brain (perhaps via the number of new connections/associations you make), which benefits you in many more ways than you first might think. I believe that engaging in all manner of creative activities has long-lasting positive effects on intellectual capability and psychological well-being, and that instead of plopping down statically to watch something, creating something keeps your brain "better-exercised", so to say. </div>
<div style="text-align: center;">
<br /></div>
Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com0tag:blogger.com,1999:blog-338911558031950356.post-68268757007416653772015-07-14T11:06:00.003-07:002015-07-14T11:07:55.398-07:00The Experiencers: this is your last jobWith the rapid growth of what A.I. is capable of, the rapid advancements of technology (via Kurzweil's Law of Accelerating Returns), the massive reach of the internet and the cloud, the obvious question concerns what the role of humans is to be when even the intellect can be mechanized? I offer my musings on a potential kind of future here: <a href="http://web.mit.edu/zoya/www/TheExperiencers_SF.pdf">http://web.mit.edu/zoya/www/TheExperiencers_SF.pdf</a>Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com0tag:blogger.com,1999:blog-338911558031950356.post-21341869298772946292015-07-02T21:57:00.000-07:002015-07-02T21:59:00.475-07:00where is innovation, and who's pulling who along for the ride?In the modern landscape of giants like Google and Facebook, and the scurry of activity generated by tech start-ups in the SF and Boston areas and beyond, one of the big questions is: where does academia sit? And how do all these forces shape each other?<br />
<br />
Big companies are no longer shaping just the industry world - they are having massive impacts on academia - both directly (by acquiring the best and brightest academics) and indirectly (by influencing what kinds of research directions get funded).<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQCKdqFuiMHBvlZOesSoiqaq54-No807xHHqG9fPLhgBCjbpSOhxbWeIfoLr4ZDEOprFe9mEJT1x0lnmjzzsPTCyHReT4wfMkyIP-wMmEBX0v5Oj8PYEZ5t69DmCfOVcqIN04XqEcZcoGQ/s1600/shaping.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="115" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQCKdqFuiMHBvlZOesSoiqaq54-No807xHHqG9fPLhgBCjbpSOhxbWeIfoLr4ZDEOprFe9mEJT1x0lnmjzzsPTCyHReT4wfMkyIP-wMmEBX0v5Oj8PYEZ5t69DmCfOVcqIN04XqEcZcoGQ/s400/shaping.jpg" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
This leaves a few hard questions for academics to think about:<br />
To what extent should industry drive academia and to what extent can academia affect where industry is going?<br />
<br />
We can follow, for instance, the big companies - sit closely on their heels, learn about their latest innovations, and project where they're likely to be 5-10 years from now. Then use this knowledge to appropriately tailor funding proposals, to direct research initiatives, and to count on the emerging technologies to fall into place. For instance, if you know that certain sensors are going to be in development in the next few years, does it not make sense to already have ready the applications for those sensors, the algorithms, the methods for processing the data? Or does this build up an inappropriate dependence, turn academics into consumers? Taking this approach, you're likely to win financially in the long run - either via funding (because your proposed projects are tangible) or via having your projects, ideas, or you-yourself acquired by the big guys (and all the advantages that go along with that). However, does this approach squelch innovation - the thinking outside-the-box, outside the tangible, and further into the future?<br />
<br />
Importantly, where is innovation coming from most these days? In one of the Google I/O talks this year, there was a projection that more than 50% of solutions will be from startups less than 3 years old in the coming future. Why is this the case? I can think of a number of reasons: bright young graduates of universities like MIT and Stanford are taking their most innovative research ideas and turning them into companies, and this is becoming an increasingly hot trend. More and more of my friends are getting into the start-up sphere, and those that aren't are at least well aware of it. Second of all, many startups are discovering niches for new technologies: whether it's tuning computer vision algorithms to the accuracy required for certain medical applications, applying sensors to developing-world problems like sanitation monitoring, or using data mining for applications where data mining has not been used before. Tuning an application, an algorithm, or an approach to a particular niche requires utmost innovation - that is where you discover that you need to use a computer vision algorithm to achieve an accuracy that was never achieved before, to create a sensor with a lifespan that was not previously imaginable, to make things work fast, make them work on mobile, make them work over unstable network connections, make the batteries last. Academically, you rarely think of all of the required optimizations and corner cases, as long as the proof-of-concept exists (does it ever really?), but in these cases, you have to.<br />
<br />
Perhaps we can think of it this way: the big guys are developing the technologies that the others do not have the resources for; the small guys are applying the technologies to different niches; and the academics are scratching their heads over application areas for these technologies and the next-to-emerge technologies - never quite rooted in the "what we have now" and always stuck (or rather, comfortably seated) in the "what if". Who's shaping who? It looks like they're all pulling each other along, sometimes gradually, other times in abrupt jerks. At any given time you might be doing the pulling or be dragged along for the ride.<br />
<br />
So where does that leave us? Are big companies, little companies, and academia taking distinctly different routes, or stepping on each other's toes? At this point, I think there is a kind of melting pot without sharp boundaries - a research project slowly transitions into a start-up, which then comes under the ownership of a big company; or a research lab that transplants its headquarters into a big company directly; or the internal organizations like the research labs or advancements labs (Google Research, GoogleX, ATAP) that have the feel of start-ups with the security and backing of a large company. It's a unique time, with everything so malleable. But I'm not sure this triangle-of-a-relationship has reached any sort of equilibrium quite yet... We have yet to wait until the motions stabilize to see where the companies and the universities stand, and whether they will continue to compete in the same divisions, or end up in vastly different leagues.<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com1tag:blogger.com,1999:blog-338911558031950356.post-20283357609622566552015-06-24T21:27:00.001-07:002015-06-24T21:30:07.723-07:00Imagining your imagination Given the news that are making such a splash recently - "dreaming A.I." and "machines with imagination" (<a href="http://globalnews.ca/news/2070281/googles-ai-creates-bizarre-yet-beautiful-images-from-its-dreams/">http://googleresearch.blogspot.fr/2015/06/inceptionism-going-deeper-into-neural.html</a>), a few interesting questions are up for pondering...<br />
<br />
An NN's (neural network's) "imagination" is a property of the data it has seen and the task it has been trained to do. So an NN trained to recognize buildings will hallucinate buildings in novel images it is given, an NN trained on YouTube videos will discover cats where no cats have ever been, etc.,.. So, an NN trained on my experience, one that sees what I see very day, (and provided it has the machinery to make similar generalizations) should be able to imagine what I would imagine, right?<br />
<br />
Facebook and Google and other social services should be jumping on this right now to offer you an app to upload all your photo streams and produce for you "figments of your imagined imagination" or "what your photos reveal about what might be in your mind" (the high-tech NN version of personality quizzes, perhaps). Basically, you can expect the output to be a bizarre juxtaposition of faces and objects and shapes (like in the news article) but customized just for you! Wait for it, I'm sure it's just around the corner.<br />
<br />
So if we strap on our GoPros or our Google Glasses and run out into the world hungrily collecting every moment, every sight, and every experience that we live through, can we then hope that our very own personal A.I.s will be able to learn from all this data to remember our dreams when we can't, guess a word off the tip of our tongue, make the same connections, parallels, and metaphors? and know what new thought our mind could have jumped to from the context of the previous conversation? As we envision that A.I. will one day augment us, do we take into account the fact that the augmentation will not be a simple division of labor: "I as the human being will leave the superior, heuristic, and creative tasks to myself, and leave my duller mechanical half to deal with all the storage and lookup and speed that I lack" -- this may be an outdated thought; perhaps "your" A.I. will be able to make bigger generalizations, leap further, find more distant connections, to innovate and create. The correct question should then be: what can YOU contribute to your A.I.?Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com0tag:blogger.com,1999:blog-338911558031950356.post-84343959903678929992015-06-18T22:02:00.002-07:002015-06-18T22:02:41.102-07:00CVPR recap and where we're goingThe Computer Vision and Pattern Recognition (CVPR) conference was last week in Boston. For the sake of the computer vision folk (at least in my group), I created a summary/highlights document of some paper selections here: <a href="http://web.mit.edu/zoya/www/CVPR2015brief.pdf">http://web.mit.edu/zoya/www/CVPR2015brief.pdf</a><br />
<br />
It takes an hour just to read all the titles of all the sessions - over 120 posters/session, 2 sessions a day, 3 days... and workshops. This field is MONSTROUS in terms of output (and this is only the 20% or so of papers that actually make it to the main conference).<br />
Thus, having a selection of papers instead of all of them becomes at least a tiny bit more manageable.<br />
<br />
The selections I made are roughly grouped by topic area, although many papers fit in more than one topic, and multiple might not be optimally grouped - but hey, this is how my brain sees it.<br />
<br />
The selection includes posters I went to see, so I can vouch that they are at least vaguely interesting. For some of them I also include a few point-form notes, which are likely to help with navigation even more.<br />
<br />
<b>Here's my summary of the whole conference:</b><br />
<br />
I saw a few main lines of work throughout this conference: CNNs applied to computer vision problem X, metric for evaluating CNNs applied to computer vision problem X, new dataset for problem X (many times larger than previous, to allow for application of CNNs to problem X), new way of labeling the data for the new dataset for CNNs.<br />
<br />
In summary, CNNs are here to stay. At this conference I think everyone realized how many people are actually working on CNNs... there have been arxiv entries popping up all over, but once you actually find yourself in a room full of CNN-related posters, it really hits you. I think many people also realized how many other groups are working on the exact same problems, thinking about the exact same issues, and planning on the exact same approaches and datasets. It's become quite crowded.<br />
<br />
So this year was the CNN hammer applied to just about any vision problem you can think of - setting new baselines and benchmarks left and right. You're working on an old/new problem? Have you tried CNNs? No? The crowd moves on to the next poster that has. Many papers have "deep" or "nets" somewhere in the title, with a cute way of naming models applied to some standard problem (ShapeNets, DeepShape, DeepID, DevNet, DeepContour, DeepEdge, segDeep, ActivityNet). See a pattern? Are these people using vastly different approaches to solve similar problems? Who knows.<br />
<br />
So what is the field going to do next year? Solve the same problem with the next hottest architecture? R-CNNs? even deeper? Some new networks with memory and attention modules? More importantly, do results get outdated the moment the papers are submitted because the next best architecture has already been released somewhere on arxiv, waiting for new benchmarking efforts? How do we track whether the numbers we are seeing reported are the latest numbers there are? Are papers really the best format to present this information and communicate progress?<br />
<br />
These new trends in the computer vision are leaving us to think about a lot of very hard questions. It's becoming increasingly hard to predict where the field's going in a year, let alone a few years from now.<br />
<br />
I think there are two emerging trends right now: more industry influence (all the big names seem to be moving to Google and Facebook), and more neuroscience influence (can the networks tell us more about the brain, and what can we learn about the brain to build better networks?). These two forces are beginning to increasingly shape the field. Thus, closely watching what these two forces have at their disposal might offer glimpses into where we might be going with all of this...<br />
<br />
<br />
<br />
<br />
<br />Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com3tag:blogger.com,1999:blog-338911558031950356.post-29227291129922276992015-06-17T00:03:00.000-07:002015-06-17T00:03:15.393-07:00The Computer History Museum in SFThe <a href="http://www.computerhistory.org/">Computer History Museum</a> in SF was great! It was a bit of a random stumble during a trip along the West Coast a few weeks ago, but it left a memorable trace! The collection of artifacts is quite amazing: name just about any time in computer history (ancient history included) and any famous computer (Babbage Engine, Eniac, Enigma, Univac, Cray, etc.) and some part of it is very likely at this museum. We totally assumed the museum would be a 2-hour stopover on the way to other SF sights, but ended up staying until closing, without even having covered all of it.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwm-Fc08QLG2eIIGw4cemLtmAwTXZE9Jwi7m38bIMoAghAZHz27dPKj7tob7uvc8w-rYdicXLiU_8xtG3fzpZoPS08kRtBcCterKejoVw6C39mP1JKZnwpmANOsQK_5KDZshdx8isFPzL7/s1600/20150529_143612.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwm-Fc08QLG2eIIGw4cemLtmAwTXZE9Jwi7m38bIMoAghAZHz27dPKj7tob7uvc8w-rYdicXLiU_8xtG3fzpZoPS08kRtBcCterKejoVw6C39mP1JKZnwpmANOsQK_5KDZshdx8isFPzL7/s320/20150529_143612.jpg" width="320" /></a></div>
<br />
<br />
As a teaser I include a few random bits of the museum that caught my attention (I may have been too engrossed in the rest of the museum to remember taking pictures).<br />
<br />
One of the oldest "computers": <a href="https://en.wikipedia.org/wiki/Antikythera_mechanism">Antikythera mechanism</a> - had never heard of it before! The Ancient Greeks continue to impress! Shows another timeless quality of humanity: our technological innovations are consistently driven by our need for entertainment (in the case of the Ancient Greeks, such innovations can be linked back to scheduling the Olympic Games). At this museum, there was a full gallery devoted to old calculators and various mechanical computing implements from different cultures.<br />
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiB3ai4EmoSVi63B0HF3sTyXQ7lWFWZexrFs2c7WqFFafiP2nUE5S5EDTLtcrk5qN1kOU-v-pC05crf6DZod42plSZDF8JH2DfbK6z1FFQNHaoLEq7wwW0i-tj6z4MeEsKlCFXXwP6gGb5B/s1600/20150529_141834.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiB3ai4EmoSVi63B0HF3sTyXQ7lWFWZexrFs2c7WqFFafiP2nUE5S5EDTLtcrk5qN1kOU-v-pC05crf6DZod42plSZDF8JH2DfbK6z1FFQNHaoLEq7wwW0i-tj6z4MeEsKlCFXXwP6gGb5B/s320/20150529_141834.jpg" width="320" /></a></div>
<div>
<br /></div>
<div>
A fully-working constructed version of <a href="http://www.computerhistory.org/babbage/">Babbage's Difference Engine</a> - completed in 2008 according to Babbage's original designs (which apparently worked like a charm without any modification!) Museum workers crank this mechanical beast up a few times a day for the marvel of the crowd. Once set, this machine can compute logarithms, print them on a rolling receipt, and simultaneously stamp an imprint of the same values into a mold (for later reprinting!) Babbage also thought of what happens when the imprinting fills up the whole mold - a mechanical mechanism halts the whole process, so that the tablet can be replaced! That's some advanced UI, developed without any debugger or user studies.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAUNbMIUBl5iYRVSO-8Hd7D8T4PbBAqCBbfEEW2ebvCjl68rriv63A9g5QbDFC2byNaVR8u5sFqX2BL8pQV_VdpOmOINVCivYULjmuBTvyyTsLixK1tGSaKCQakOkGGlL5THZnqwQlki5F/s1600/20150529_130239.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAUNbMIUBl5iYRVSO-8Hd7D8T4PbBAqCBbfEEW2ebvCjl68rriv63A9g5QbDFC2byNaVR8u5sFqX2BL8pQV_VdpOmOINVCivYULjmuBTvyyTsLixK1tGSaKCQakOkGGlL5THZnqwQlki5F/s320/20150529_130239.jpg" width="320" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhu44TabcKRt8NA_uPIc7ZI4HJ_qL2vIt3-J-g3BhWDhoNAsKdoAXWdy90VLRICS9wIibIjb9tJrly0XiWymiK8Yf1ykWFfR8MVOXlWR_g4XwcfRM7zAnAtNhuURFMZsFfJPmMCYi8dsVzW/s1600/20150529_131200.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhu44TabcKRt8NA_uPIc7ZI4HJ_qL2vIt3-J-g3BhWDhoNAsKdoAXWdy90VLRICS9wIibIjb9tJrly0XiWymiK8Yf1ykWFfR8MVOXlWR_g4XwcfRM7zAnAtNhuURFMZsFfJPmMCYi8dsVzW/s200/20150529_131200.jpg" width="112" /></a></div>
<div class="separator" style="clear: both; text-align: justify;">
Based on the over-representation of this Babbage Engine in this post, you can tell that quite a bit of time was spent gawking at it:</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /><iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/sI1izkDBb8E/0.jpg" src="https://www.youtube.com/embed/sI1izkDBb8E?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
By the way, here's a real (previously functional) component from the Univac. Unlike that panel with lights and switches at the top of this post. Apparently, that did not do anything. It was purely for marketing purposes for whenever the then-investors came around to check out this "machine" - much more believable that something real is happening when you have dashboard blinking of some kind and large buttons that serve no purpose but look "very computational". Looks like this continues to be a powerful marketing strategy to this day :)</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkeVtDctHUh8w0lgBiUhuCPimMS3wQznEhjH4C7_dEvOiJWQh_hGCCvZfi5QAdA_sYY2H5brJztcmxFdWY6W48qfIgEEXzGn1pWU3vQycgejf97s99xWmO_UGTHhJMNXmKQiYUAEcaPE7r/s1600/20150529_143618.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkeVtDctHUh8w0lgBiUhuCPimMS3wQznEhjH4C7_dEvOiJWQh_hGCCvZfi5QAdA_sYY2H5brJztcmxFdWY6W48qfIgEEXzGn1pWU3vQycgejf97s99xWmO_UGTHhJMNXmKQiYUAEcaPE7r/s320/20150529_143618.jpg" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
Just a fun fact (no, this is not the origin of the word "bug", which is what I thought this was at first, but does demonstrate some successful debugging):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi9QQaGIMeVDGAWyOFfRrS8dCOhEgLWMVJnY7O9epbcSFiI5LPGJatNsx1SijZbJ6NBbRGmodyTQUhHi4DXx82Nm5I5j9LFSHRaGzizjhPTAhijXuGPgRRlk8cZ63WDkkDJsV1LNythKSFK/s1600/20150529_153839_001.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi9QQaGIMeVDGAWyOFfRrS8dCOhEgLWMVJnY7O9epbcSFiI5LPGJatNsx1SijZbJ6NBbRGmodyTQUhHi4DXx82Nm5I5j9LFSHRaGzizjhPTAhijXuGPgRRlk8cZ63WDkkDJsV1LNythKSFK/s320/20150529_153839_001.jpg" width="320" /></a></div>
<br />
The following describes quite a few computer scientists I know:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNJ9c0yiXwyk3DlFv3TFOtSoIKa0j4gWrJnXRk77V5ndx4V0IRtsoEyctZXAKfib6u26RTvuLrtDSj0vgy378n5llUYqP1NxqcoX1X8FEMQ2ekcbUpwGq9HE7eckoNmVYDw6HzSF55WC4g/s1600/20150529_160440.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNJ9c0yiXwyk3DlFv3TFOtSoIKa0j4gWrJnXRk77V5ndx4V0IRtsoEyctZXAKfib6u26RTvuLrtDSj0vgy378n5llUYqP1NxqcoX1X8FEMQ2ekcbUpwGq9HE7eckoNmVYDw6HzSF55WC4g/s320/20150529_160440.jpg" width="320" /></a></div>
<br />
There was a gallery devoted to Supercomputers and another gallery devoted to computer graphics. Look at <a href="http://www.sjbaker.org/wiki/index.php?title=The_History_of_The_Teapot">what</a> I found there - every Graphics PhD student's rite of passage (by the way, the <a href="http://www.cs.toronto.edu/~rgrosse/intrinsic/gallery.html">Intrinsic Images dataset</a> is sitting in my office, no glass case, but we will soon start charging chocolate to see it):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjj7UuWjxg-G8rEVgD-zg0CfHk2eaclj1EGjkBRyDOY5l6VdjFm-vENPp07dPSEwlnNILfIC-V_5Gc70q4xccddDB4qFpxRRe1UjAQx-88nmhsRJkyIv9VcKXB-sP1PcC1Q9-K6Kd1f3A6Q/s1600/20150529_162511.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjj7UuWjxg-G8rEVgD-zg0CfHk2eaclj1EGjkBRyDOY5l6VdjFm-vENPp07dPSEwlnNILfIC-V_5Gc70q4xccddDB4qFpxRRe1UjAQx-88nmhsRJkyIv9VcKXB-sP1PcC1Q9-K6Kd1f3A6Q/s320/20150529_162511.jpg" width="180" /></a></div>
<br />
There was also a whole gallery devoted to robots and A.I. (an impressive collection), a gallery devoted to computer games, and a gallery devoted to the Apple computer just to name a few.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKWSQT-luwCjuLCnrDdrYDFlXYgSotQK6JegvupJXwQRJzsS2Ll8o1G8voP25Q0zDwArFqJfvD-7Dt5aESRWD3LQiuVc8Hy7BtnLgVx-bOMMmupqySWNl9AtiHPh-wEawX1epn-dGA5T82/s1600/20150529_162644.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKWSQT-luwCjuLCnrDdrYDFlXYgSotQK6JegvupJXwQRJzsS2Ll8o1G8voP25Q0zDwArFqJfvD-7Dt5aESRWD3LQiuVc8Hy7BtnLgVx-bOMMmupqySWNl9AtiHPh-wEawX1epn-dGA5T82/s320/20150529_162644.jpg" width="320" /></a></div>
<br />
By the way, something I didn't know about the Apple computer - here is some awesome bit of marketing that came out in 1984: <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/axSnW-ygU5g/0.jpg" src="https://www.youtube.com/embed/axSnW-ygU5g?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<br />
<div class="separator" style="clear: both; text-align: justify;">
There was a gallery devoted to the Google self-driving car. I like how this is in the Computer History museum, because really, you can't put any computer technology in a museum and assume it will remain current for very long. The drone in the corner of that room had a caption that mentioned something about possible future deliveries. Old news. I've seen bigger drones :) </div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjtFclrvOxYIWiR6UIw8YnQZPLFMAwhMzP0X7Zu3HqOrbGkYIEzP-ecOIZmtjjp5AYB51kEBMsvwbt3n825KLtGthrD3g5ZeoIuaCIyhOFz_KbmPeSFk9X-zoYRftx6Hby-OcEmFH2g9msZ/s1600/20150529_135257.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjtFclrvOxYIWiR6UIw8YnQZPLFMAwhMzP0X7Zu3HqOrbGkYIEzP-ecOIZmtjjp5AYB51kEBMsvwbt3n825KLtGthrD3g5ZeoIuaCIyhOFz_KbmPeSFk9X-zoYRftx6Hby-OcEmFH2g9msZ/s320/20150529_135257.jpg" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
That's about the extent of the photos I took, because photos really fail to convey the environment that a museum surrounds you with. It is a museum I would gladly recommend!</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
As an after-thought, it's interesting to visit a "history" museum where you recognize many of the artifacts. Gives you a sense of the timescale of technological innovation which continues to redefine what "history", "progression" and "timescale" really mean... notions that we have to regularly recalibrate to.</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div>
<br /></div>
Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com0tag:blogger.com,1999:blog-338911558031950356.post-42549307694323946212015-06-13T09:38:00.001-07:002015-06-13T09:39:18.437-07:00Google I/O RecapAnnouncements from Google I/O are increasingly popping up over the media.<br />
Last year, after going to Google I/O I compiled a series of slides about some of the top announcements and some of the other sessions I went to: <a href="http://web.mit.edu/zoya/www/googleIOrecap.pdf">http://web.mit.edu/zoya/www/googleIOrecap.pdf</a><br />
This year, I watched many Google I/O videos online, and I've compiled a small summary here: <a href="http://web.mit.edu/zoya/www/googleIO2015_small.pdf">http://web.mit.edu/zoya/www/googleIO2015_small.pdf</a><br />
As a researcher, I find it instructive to look to where such giants such as Google are moving in order to get a sense of which research directions and developments will be especially in need in the coming while. Thus, I look at the talks from an academic perspective: what are the key research questions surrounding every product? I tried to include some of these in my latest slides.Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com0tag:blogger.com,1999:blog-338911558031950356.post-66695703822774823662015-06-02T14:49:00.001-07:002015-06-02T14:49:07.829-07:00Why Google has the smartest business strategy: openness and the invisible workforceGoogle works on an input/output system. In other words, for everything that Google developers create, Google accepts input from users and developers around the world. Note that the latter group/community is orders of magnitudes larger than the former, so by harnessing the resources and power from the users and developers around the world, Google's Global footprint becomes significantly larger.<br />
<br />
For instance, Google produces continuos output in the form of products and developer platforms, and accepts input in the form of development directions and most importantly, apps. By creating platforms on which developers can build on top of, Google harnesses the users that want the apps. The more that Google releases (e.g. SDKs), the more developers are looped in to create new apps, and the more users get pulled in to use the apps, thus acquiring the Google products in the process. Thus, the number of people around the world that are increasing the consumer base for Google products far extends past the number of Google employees.<br />
<br />
In fact, the number of people indirectly working for Google is huge. Consider the <a href="https://developers.google.com/groups/">Google Developer Groups (GDGs)</a> that can be found all around the world - independent organizations of developers and enthusiasts that get-together to bond over Google's technology (they also give Google product-related talks and host help sessions for their local communities, all on their own time). What's in it for the members? Members of GDGs have the support and network of individuals with similar interests. Google wins by having a Global network of communities that are self-sufficient and self-reinforcing and do not require Google support or investment. <a href="http://googlesystem.blogspot.ca/2013/03/google-trusted-testers-from-betas-to.html">Google Trusted Testers</a> are non-employees that test beta products for Google. What's in it for the testers? First-hand experience with Google products. What's in it for Google? A workforce for whom being "first to try a product" is sufficient reward. The <a href="https://www.google.com/edu/resources/programs/student-ambassador-program/">Google Student Ambassador Program</a> gives college students an opportunity to exhibit leadership by acting as a liaison between Google and their home institution, putting on Google-supported events (information sessions, hackathons, etc.) and forming student communities. The student ambassador's motivation is a nice line on their resume and great experience communicating with both industrial and institutional personnel and organizing events. Google wins by being promoted on college campuses and having easier avenues for student recruitment... all for the price of providing some Google-themed freebies at college events. Then there's all the other smaller organizations that are not directly supported by, but have affiliation with, Google. For instance, the <a href="https://www.facebook.com/pages/Anita-Borg-Scholarship-Alumni-Community/261363020730608">Google Anita Borg Alumni Planning Committee</a> that I am part of is devoted to increasing visibility and interest in computer science among minorities and help promote diversity in computer science education. We, as a group of females distributed Globally, start initiatives and put on events (<a href="http://web.mit.edu/cs-visit-day/index.html">such as the following</a>) in our local communities to advance these missions. Google provides the branding. We win through affiliation with Google, Google wins through affiliation with philanthropic organizations. These are just a few of the organizations and communities that are affiliated with but not directly supported (at least financially) by Google. In fact, Google does not need to directly support or control/govern any of these communities precisely because they are self-sufficient and self-motivated - a big win for Google, given the limited investment.<br />
<br />
Now consider the yearly Google I/O conference that draws over 5,000 attendees. Many of these attendees are developers who come to the conference to hear first-hand about new product and platform releases (and participate in hands-on workshops with the Google product developers themselves). These developers then bring this knowledge back to their communities, and contribute their own apps and products to the Google community. Each year, at this conference, Google announces new support infrastructures to make the use of Google products increasingly easier (this year, for instance, Google announced new OS and language support for the <a href="http://www.theverge.com/2015/5/28/8677119/google-project-brillo-iot-google-io-2015">Internet of Things</a> so that developers can more easily add mobile support to physical objects - think: the smart home). Correspondingly, the number of Google product-driven apps increases and expands. Users of apps buy Google products and services and continuously provide feedback (either directly through surveys or indirectly by having their interactions and preferences logged on Google servers). Thus, we are all contributors to the growth of the Google footprint.<br />
<br />
What can we infer from all of this? Google is firmly rooted in our societies and is here to stay. The number of people supporting, improving, and building on top of Google products is huge - it is Google's invisible workforce. Thus, Google will continue to grow and improve at great speeds.<br />
<br />
What lesson can we learn from all of this? Being open (i.e. in terms of software and even hardware) can allow a company to harness the power of other developer and user communities, thus increasing the size of the effective workforce that builds the company's products, directions, and reputation. Google has one heck of a business strategy.<br />
<br />
<br />Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com0tag:blogger.com,1999:blog-338911558031950356.post-64580958706369175942015-05-31T13:48:00.002-07:002015-05-31T13:48:55.856-07:00Freeing humans from redundancyThis has been, and should be, the ultimate goal of mechanizing and automating the world around us. The human mind is too precious of a resource to waste on any types of repeating and repeatable actions, and we've been working on automating the latter since the Industrial Revolution. With modern A.I., more of this is becoming possible.<br />
<br />
Consider showing a robot once how to clean the window - defining the physical boundaries, and indicating preferences for the action sequence: you can perform this specification of actions once, the robot repeats this for a specified period of time (e.g. an hour), repeating this sequence at regular intervals (say, once a week). Consider, in such a way, seeding a large variety of actions - watering plants, making mashed potatoes for dinner, tuning the bike, drycleaning your suits, etc. I am not imagining a single robot with the A.I. to do all these tasks (not to mention knowing when to do them) - I am rather imagining an army of simple machines that can be individually programmed by their owners (programming not in the coding sense, but in the show-by-example sense).<br />
<br />
You put on your VR (virtual reality) headset while sitting on your hotel bed in SF, you log into your Boston home, seeding a bunch of actions through the FPV cameras on your machines (machine 1 will water the plans on your balcony, machine 2 will set some bread baking, machine 3 will scan some documents for you after finding them in the relevant folders on your shelf). You do the seeding for your cottage country house on Long Island as well. In such a way, not moving off your bed, you have now prepared your Boston home for your arrival tomorrow, and have checked in on your cottage. Here's the critical point: none of these machines has had to be hard-wired for your house or for any of the actions you have assigned them - they simply have the capacity to learn an action and a schedule for it (which does not require any complex A.I. and is completely feasible already). It is up to you to make the difficult human decisions of setting the schedule - when and how much to water the bonsai and the petunias, how long to wait before the bread has just the right crust for your taste, which of your clothes need special attention during dry cleaning, etc. Then the machine executes a repeatable sequence. If a condition arises for which the machine requires a decision to be made, you are pinged. Next time this condition arises, the machine has a stored solution. With time, your army of machines has been customized to all your preferences, and you have been taken out of the loop for anything that does not require an expert opinion (yours) or an indication of specific preferences (yours as well). Your mind becomes freed from anything at all repeatable or redundant, with its capacities available for the decision making and heuristic decisions that are the hallmark of human intelligence. You spend your time delivering instructions and managing outputs with utmost efficiency.<br />
<br />
I think we will much sooner see this type of future with simple customizable learning agents than one with the courteous all-in-one robotic butler you see in the movies. In fact, you can already remotely control your house heating via Nest and your music system via Sonos, as two examples, all from your Android devices (cell phone, watch). The next step is simply the augmentation of your control options (from clicks and menus) to actions with as many degrees of freedom as your arm gestures permit via a VR device. This puts a larger portion of your household devices and tasks at your virtual fingertips. The Internet of Things is here.Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com0tag:blogger.com,1999:blog-338911558031950356.post-62132026920632388732015-04-30T15:17:00.001-07:002015-04-30T15:17:51.362-07:00Filling the internet with more cool scienceSo the second year of the extreme science SciEx video competition has come to an end, with a new series of cool videos to show for it: http://sciex.mit.edu/videos/. The goal of this initiative is to take more direct steps towards making science and engineering catchier and more (socially) shareable than say... videos of cute kittens. Videos related to science should not be left to sit in some isolated corner of the internet, to be found only by people who were already looking for them... rather, they should be sprinkled around in every shape and form - extreme, cool, artistic, you name it. They should be able to affect, in some positive way, all sorts of individuals with diverse interests and personalities. Why? Because science teaches us to think, and individuals capable of thinking makes society a better place. More sharing of science will lead to a greater appreciation, at least a little bit of extra understanding, and a potential reduction of ignorance, about science. If nothing more, by actively sharing science we promote positive attitudes towards it (even if we don't always succeed at longer-ranged interest)... and positive attitudes lead to positive change... in people, in society, and in government. We need to educate the next generation of scientists, and we can contribute to this mission in the subtlest of ways and by taking small steps - including having more science and engineering videos circulating on the web. Let's show the world (the next generations) that science can be as extreme as an extreme sport, as beautiful as a work of art, as releasing as a dance. Let's celebrate scientists and engineers for the rockstars that they are. Let's cheer for them louder than we cheer for football players - because they are the ones changing the world we live in.<br />
<br />Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com0tag:blogger.com,1999:blog-338911558031950356.post-30603900668036196192015-02-15T10:24:00.003-08:002015-02-15T10:24:41.830-08:00Educational nuggetsQuite a while back I attended an MIT Task Force Retreat on Digital Learning. Numerous talks and discussions were given (by various internal MIT groups and committees) about the future of online education and the issues surrounding it. One concept that stood out to me was that of "educational nuggets". I think this is very suitable (and sticky) terminology to describe the bite-sized educational modules - like the 5-10 minute lectures - that have become a popular medium for online courseware and educational websites such as Khan Academy.<br />
<br />
The idea of bite-sized lectures comes from educational research showing that a student's attention span does not extend much, past about 10 minutes. A sad truth. That is not to say, however, that a student can not internalize concepts past 10 minutes (if that was really the case, then school systems would not work at all). Rather, efficiency of learning goes down, and more mental effort needs to be expended to stay attentive - instead of, say, all that mental effort being channelled to learning the concepts.<br />
<br />
So, it seems to be most effective to present material to students for about 10 minutes at a time, and then break up the stream by giving students some time to think about the concepts, asking students questions (or asking for their questions), providing a quiz module, initiating a discussion (if applicable), etc. This allows students to more actively internalize the material, apply the concepts, and check that they have understood the past 10 minutes worth of content.<br />
<br />
The additional advantage of splitting educational material into nuggets is to break up a course into little, self-encapsulated, independent units. To go back to the previous post, this provides a means for customization: both for the individual student, and for the individual course. Imagine the course of the future: you are a biologist looking to brush up on statistics. Instead of pointing you to a full course offered by the statistics department, or instead of having to specifically design a course on statistics in the biology department, you could be given a set of "nuggets" to complete. These nuggets could come from different places - from the statistics department, from the math department, from the biology department - such that when they all come together, they give you - the biologist - the statistics knowledge you need, in the right context, with maximal relevance.<br />
<br />
The concept of educational nuggets naturally raises some questions: is everything really nugget-izable? what about basic courses like calculus that need to be taken in full? who will decide what goes into a nugget? can many small nuggets really be equivalent to a course?<br />
I think if we become more accepting of this form of education and the benefits that we can glean from it, then the answers to these questions will start to emerge through discussions.<br />
<br />
The bigger philosophical question is whether we are changing too much, as a human species, and becoming too ADD with all the bite-sized facts, bite-sized tweets, bite-sized news, and potentially bite-sized education thrown at us. Like many things, this is a two-edged sword -- and like the related notion of multitasking, can either make or break productivity and long-term memories and understanding. The related benefits/downsides of multitasking will be left for a future post...Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com0tag:blogger.com,1999:blog-338911558031950356.post-49580154724946502282015-02-08T10:17:00.000-08:002015-02-08T10:17:10.888-08:00Education as customized paths through knowledge graphsLately very frequently I've been involved with, and witness to, discussions about the upsides/downsides of online learning in comparison to traditional classroom learning. I'd like to summarize a few of my main views on this point.<br />
<br />
The traditional classroom has a 1:N ratio of teachers to students, where N grows large for many basic-level courses. Tutoring can provide a 1:1 ratio, and has been found (by multiple quantitative studies) to be more successful at getting concepts across to students. Why? Tutoring provides customization to the individual, and thus can build off of the knowledge base of that individual. New information can hook onto any existing understanding the individual already has, and <u>this is what can let the concepts stick</u>. New concepts become more tightly intertwined with what the individual already knows (and perhaps cares about), and are thus more relevant than concepts presented in the most general setting, with no customization.<br />
<br />
In a recent talk of Peter Norvig's that I went to (Norvig is originator of MOOCs: massive open online courses), he indicated that even artificial tutoring systems can have the same benefits as human tutors, with statistically-significant benefits over the traditional classroom. This is very promising, because artificial tutoring is a potentially infinite resource (unlike the finite number of good-quality human tutors). In the same talk, Norvig put up a slide of a dense knowledge graph of all the information that can be available to a student on a particular topic in a particular course(s). He drew some squiggly lines through this graph, standing in for unique paths that could be taken through that material. This is the same visual representation of customized learning that I envision, and deeply believe in, for the future of education.<br />
<br />
There is no reason why different individuals should take the same paths through learning. Different types of information may be relevant to different people, and a different ordering of material may make more sense to some individuals but not others. Naturally it should be possible to constrain which points an individual should definitely pass through for a particular course/subject matter (to cover the fundamentals), but the paths themselves should be less constrained. This is the diversity that I referred to in my previous post, which is why I believe that online education is the way forward.<br />
<br />
We already have almost all the tools to make this a reality: (1) sophisticated machine learning algorithms that can pick up on trends in user data, detect clusters of similarly-behaving individuals, and make predictions about user preferences; (2) thorough user data through logging and cloud storage, integration of physical and virtual presence and social networks, integration of all of a user's applications and data (and the future of the "internet of things"), universal login systems, etc.<br />
<br />
Thus, the question is only one of time.<br />
<br />Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com0tag:blogger.com,1999:blog-338911558031950356.post-84832657881248171852015-02-03T18:21:00.001-08:002015-02-03T18:21:23.021-08:00The Popping Rate of KnowledgeI use the term "popping rate" to refer to the amount of novel/interesting/useful material gained in a given time period. If you're used to microwaveable popcorn, you know that as the popping rate decreases past a certain point and pops become rarer - if you don't take the popcorn out of the microwave, it will fry. I think the same goes for my brain when it is trying to suck up knowledge. If I'm watching a really interesting documentary, reading a good nonfiction book, or listening to a captivating talk, I can almost feel the new knowledge and facts pop and fill my brain. I consider the time well-spent if the popping rate is above a certain threshold... however, if the pops become too rare, I feel my brain frying under lack of stimulation. That is when I know to turn off the TV, put down the book, or zone out of the talk... and pursue an activity with a higher popping rate.<br />
<br />
In fact, I've found that quantifying the informational/factual content of something using some notion of pops per minute or pops per hour (referring to # of novel bits of information/facts gained during that time), provides a useful frame of comparison between activities.Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com0tag:blogger.com,1999:blog-338911558031950356.post-40684216541218400512015-01-31T09:25:00.000-08:002015-01-31T09:25:35.099-08:00Increasing diversity in computer scienceI recently organized a panel of MIT computer science researchers to answer questions about computer science to an audience of high-schoolers (http://web.mit.edu/cs-visit-day/qa.html). A lot of interesting discussions came out of it, and a lot to digest for panelists and audience alike.<br />
<br />
One of the things that did stick out to me was how many of the panelists did <u>not</u> like their first formal training in computer science (in school, in college). I can't say I was terribly surprised, and I'd be interested to see such a survey done of the broader CS community (e.g. at a research institution) to poll about initial experiences and attitudes. <br />
<br />
Here is where I think the problem lies: computer science courses tend to cater to a very narrow audience - maybe the type of audience that likes computer/video games, or the type of audience that likes tech gadgets, etc. Not that you can avoid it: you have to start <i>somewhere - </i>with some (salient) example or application or first program. But once you've settled on something, you might get one group of individuals hooked, but you'll also automatically repel a lot of other people (who are not interested in the application area you've chosen). If that was the only/first opportunity those people had to learn computer science, they might decide they don't like it and never pick it up again - which would be an enormous shame!<br />
<br />
What's the solution? Introducing more variety and choice into the computer science curriculum - tailoring it to different tastes (and personalities!). Cater it to people who might like biology or psychology or architecture or design, and show them that computer science can provide them with a toolset, a simulation/virtual environment to test their ideas, a cool exploratory possibility. I believe this is the way forward to increasing the <u>diversity of people</u> in the field of computer science.<br />
<br />
In practice, having a lot of variety in a computer science curriculum may not be possible (consider a school with a single programming course and a single teacher to teach it)... in this case, I think online education with its possibilities for individual customization, can come to the rescue... more about this later.Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com6tag:blogger.com,1999:blog-338911558031950356.post-86486433714476006662014-11-12T16:42:00.002-08:002014-11-12T16:42:43.487-08:00Externalizing creativityTo follow-up on my previous post, I watched a relevant Ted talk the other day: http://www.ted.com/talks/elizabeth_gilbert_on_genius<br />
<br />
A popular writer describes why being an artist, or more generally, being creative is so hard on a person - you get subjectively evaluated for your creations, you have to worry about not having enough creative ideas to fuel you further, you continuously have to compete with yourself (specifically your own creativity and imagination). And all of this can lead to depression (a depressed creative genius is not a rare sight).<br />
<br />
She mentions that the ancient Greeks and Romans got it right: they attributed creativity to an external spirit that either did a good job or not. This is the type of thinking that could shield an artist from his creative genius, making creativity an external factor. A successful artist need not suffer from narcissism, an unsuccessful one need not blame himself (rather, blame the lame spirit).<br />
<br />
I came to the same conclusions about the difficulty of being a grad student, and for the same reasons. Grad students are creative individuals, that first have to conjure up new problems... and then, solve them. Learning to separate oneself from one's ideas can do wonders for sanity.<br />
<br />
Here's a possible approach: when you come up with an idea, step aside from it (go as far as sending yourself an e-mail with the idea or leave an "anonymous" note on your own desk). Pick up the note, read the idea, critique it (praise the idea, or politely refuse to follow through with it). If the idea doesn't end up succeeding in the long run, shrug your shoulders: the anonymous note writer might leave you a better idea next time...<br />
<br />
----<br />
<i>post scriptum:</i> I often catch myself unable to fall asleep because all the good ideas start coming at 1 am. There's many of them, and they're all scattered. Paralleling Liz Gilbert's story about the poet on the road, I want to shake my fists at all this "external" creativity and yell: "just let me sleep once in a while, will ya? come back tomorrow morning when my brain can sort through all this")Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com0tag:blogger.com,1999:blog-338911558031950356.post-25222579507347753822014-11-12T16:09:00.000-08:002014-11-12T16:09:04.435-08:00The double-think game of being a grad studentMany people have written about grad school, often in a comical (even satirical) light. Perhaps I will leave my own detailed grad-school comments for after I graduate (it's safer that way :))<br />
<br />
But my view of why it can be so difficult on a person, is that the issues faced and the pressures that exist are more often internal, rather than external. And internal issues are the worst kind of issues to deal with. Why? you are your own boss, you know all the weak spots and how to press on them. Probably no one can be as severe to you as you can be to yourself. Nothing you do is enough.<br />
<br />
When an issue/pressure is external, you can distance yourself from it. An internal issue eats you from the inside, and there's no hard shell you can put up in your defense.<br />
<br />
As a grad student you have to engage in a very difficult kind of double-think game: you have to care enough about your problems to push through them, but you have to know how to distance yourself from them at the right time, not dwell, and not let them get to you.<br />
<br />
The good news is that grad students are the types of individuals that love challenges, and learning to play this double-think game is just one more challenge to be conquered.<br />
<br />Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com0tag:blogger.com,1999:blog-338911558031950356.post-40779100617148890742013-10-14T09:36:00.001-07:002013-10-16T18:57:05.177-07:00Rabbits have good eyesight... they don't stare at computersIt is no wonder that when spending most of the day staring at the computer, ones eyes get progressively worse at seeing distant objects: after all, you are increasingly training your eyes to read up close, and losing the need to see something far away. Recently, this fact hit me hard when I went to the grocery store and realized I could no longer make out the aisle banners. Those banners are made for average people, with average eyesight, and I am not used to being below average!<br />
<br />
It turns out that exercising your eyes regularly can help prevent some deterioration (or at least the type that is caused by spending most of the time in close focus, without training a farther focus). For some ideas, check out this page: http://www.wikihow.com/Exercise-Your-Eyes<br />
<br />
In an effort to train my own eyes, I covered my office with those eye charts you would see at the eye doctor's. I put these on multiple walls, at different distances and different heights, all around my office, and in the corridor across from my office. I now force myself to take regular break from my computer, to look up and read out some of the letters on the various walls.<br />
<br />
When there is no horizon outside your window (but the wall of the nearest building), you have to improvise and create your own horizons to train your eyes on.<br />
<br />
It is ok that I eventually learn which letters are on which walls - because the point is not to to read them out from memory, but to force one's eyes to follow the letter contours. Even if I know there's an "E" there, it's hard to find all its pieces.<br />
<br />
Ever since starting this, I've started noticing my eyes getting tired at staring at the screen (something I haven't noticed, or probably ignored previously) - a sign that my eyes know they should be doing something else as well. Moreover, I can track my progress by noticing which letters my eyes can make out. Similarly, I can see how my eyesight degrades throughout the day (as my eyes get more tired), if I don't do the exercises.<br />
<br />
In any case, even if this type of eye training will not save me from the expected deterioration due to excessive computer usage, at least the regular breaks of looking away from the monitor will help my sanity.<br />
<br />
<br />Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com0tag:blogger.com,1999:blog-338911558031950356.post-18415955689509878182012-09-17T16:00:00.000-07:002012-09-17T16:01:04.884-07:00"the web, the chain, the tree"Last week I went to a talk by Steven Pinker on elements of style (<a href="http://web.mit.edu/nse/events/communicating-science-and-technology.html">http://web.mit.edu/nse/events/communicating-science-and-technology.html</a>). Of the whole talk, what I found most memorable was his description of the data structures for the representation of information: as he puts it - 'the web, the chain, the tree'. Our ideas are stored as an interconnected web, and we are often faced with the dilemma of writing them down in a linear structure (one sentence at a time, transitioning between at most two ideas: the previous, and the current sentence). To do this, we arrange our web into a hierarchy of ideas, which then dictates the overall order in which we can express it. In other words, we go from web to hierarchy, and then parse the hierarchy to arrive at a linear structure which we can put down on paper. This is a great way to put it! I've always been faced with the dilemma of wanting to say too many things at once, transitioning from any one idea to ten others, but have always been constrained by the necessary linear ordering of paragraphs. By the time you're done explaining the connection between the first and second ideas, you've lost a possible transition from the first to the third ideas (often requiring a return to the first idea and a new transition). I don't think this linear structure serves as a great representation of the ideas I want to express - I'd much rather hand in my essays in web form...Zoyahttp://www.blogger.com/profile/14823753189268642715noreply@blogger.com0