-
Notifications
You must be signed in to change notification settings - Fork 1
/
getting-aws-setup-and-the-basics-of-attention.html
executable file
·160 lines (123 loc) · 8.25 KB
/
getting-aws-setup-and-the-basics-of-attention.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
<!DOCTYPE html>
<html lang="english">
<head>
<link href='//fonts.googleapis.com/css?family=Source+Sans+Pro:300,400,700,400italic' rel='stylesheet' type='text/css'>
<link rel="stylesheet" type="text/css" href="http://pmlg.github.io/theme/stylesheet/style.min.css">
<link rel="stylesheet" type="text/css" href="http://pmlg.github.io/theme/pygments/github.min.css">
<link rel="stylesheet" type="text/css" href="http://pmlg.github.io/theme/font-awesome/css/font-awesome.min.css">
<link href="http://pmlg.github.io/feeds/all.atom.xml" type="application/atom+xml" rel="alternate" title="Perth Machine Learning Group Atom">
<meta charset="utf-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<meta name="HandheldFriendly" content="True" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="robots" content="" />
<meta name="author" content="PMLG" />
<meta name="description" content="Great turn out this week and fantastic to see so many new faces. For Session One this week Alan did a great job of talking us through his trails and tribulations in getting AWS setup from a windows platform. The key takeaways were: Use the Windows Bash shell as opposed …" />
<meta name="keywords" content="">
<meta property="og:site_name" content="Perth Machine Learning Group"/>
<meta property="og:title" content="Getting AWS setup and the basics of attention"/>
<meta property="og:description" content="Great turn out this week and fantastic to see so many new faces. For Session One this week Alan did a great job of talking us through his trails and tribulations in getting AWS setup from a windows platform. The key takeaways were: Use the Windows Bash shell as opposed …"/>
<meta property="og:locale" content="en_US"/>
<meta property="og:url" content="http://pmlg.github.io/getting-aws-setup-and-the-basics-of-attention.html"/>
<meta property="og:type" content="article"/>
<meta property="article:published_time" content="2017-06-08 00:00:00+08:00"/>
<meta property="article:modified_time" content=""/>
<meta property="article:author" content="http://pmlg.github.io/author/pmlg.html">
<meta property="article:section" content="events"/>
<meta property="og:image" content="http://pmlg.github.io/images/network.png">
<title>Perth Machine Learning Group – Getting AWS setup and the basics of attention</title>
</head>
<body>
<aside>
<div>
<a href="http://pmlg.github.io">
<img src="http://pmlg.github.io/images/network.png" alt="" title="">
</a>
<h1><a href="http://pmlg.github.io"></a></h1>
<h2>Perth's Machine Learning Group</h2>
<p> We are a group of students, industry professionals, entrepreneurs and curious individuals studying machine learning. We run beginner and advanced sessions every Thursday from 6-8pm at Flux, 191 St George's Terrace.</p>
<nav>
<h3>Links</h3>
<ul class="list">
<li><a href="https://groups.google.com/d/forum/perthdeeplearners" target="_blank">Join Our Mailing List</a></li>
<li><a href="http://course.fast.ai/" target="_blank">Deep Learning Course Material</a></li>
</ul>
</nav>
<ul class="social">
<li><a class="sc-GitHub" href="http://github.com/pmlg/" target="_blank"><i class="fa fa-GitHub"></i></a></li>
</ul>
</div>
<div>
<iframe src="https://calendar.google.com/calendar/embed?title=Our%20Events&showNav=0&showDate=0&showPrint=0&showTabs=0&showCalendars=0&showTz=0&mode=AGENDA&height=400&wkst=1&bgcolor=%23FFFFFF&src=lr71qjb5phpmm9ppss0cngh9jo%40group.calendar.google.com&color=%23125A12&ctz=Australia%2FPerth" style="border:solid 1px #777" frameborder="0" scrolling="no"></iframe>
</div>
</aside>
<main>
<article class="single">
<header>
<h1 id="getting-aws-setup-and-the-basics-of-attention">Getting AWS setup and the basics of attention</h1>
<p>
Posted on Thu 08 June 2017 in <a href="http://pmlg.github.io/category/events.html">events</a>
</p>
</header>
<div>
<p>Great turn out this week and fantastic to see so many new faces.
For Session One this week Alan did a great job of talking us through his trails and tribulations in getting AWS setup from a windows platform. The key takeaways were: Use the Windows Bash shell as opposed to cygwin and how transfer learning can reduce the amount of computation you need to perform, meaning that you can run these models just on your PC, albeit slowly. Finally we concluded can that for lesson 1 probabilistically it can be hard to tell a cat from a plastic bag.</p>
<p>For Session Two, the more advanced session, after many AV issues we only had time for one talk on the "basics of attention" We covered at a high level attention for context and location, soft and hard attention. The talk assumed a knowledge of lstm sequence to sequence models and showed, hopefully, seq - seq encoder decoder networks use attention to define context and location for nlp and vision.
The references for this talk are below:</p>
<p>Next week:
Sessions one will cover lesson three as group learning session.
Session Two will he a talk on Embeddings by Michael.</p>
<h1>References:</h1>
<p><a href="http://distill.pub/2016/augmented-rnns/">Attention and Augmented Recurrent Neural Networks
Chris Olah, Shan Carter</a></p>
<p><a href="https://arxiv.org/pdf/1508.04025.pdf">Effective Approaches to Attention-based Neural Machine Translation
Minh-Thang Luong Hieu Pham Christopher D. Manning</a></p>
<p><a href="https://arxiv.org/pdf/1502.03044.pdf">Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Yoshua Bengio et al.</a></p>
<p><a href="https://devblogs.nvidia.com/parallelforall/introduction-neural-machine-translation-with-gpus/">Kyunghyun Cho p1</a>
<a href="https://devblogs.nvidia.com/parallelforall/introduction-neural-machine-translation-gpus-part-2/">p2</a>
<a href="https://devblogs.nvidia.com/parallelforall/introduction-neural-machine-translation-gpus-part-3/">p3</a></p>
<p><a href="https://www.youtube.com/watch?v=omHLeV1aicw">vid</a>
<a href="https://www.slideshare.net/xavigiro/deep-learning-for-computer-vision-attention-models-upc-2016">attention models</a></p>
<p><a href="https://www.youtube.com/watch?v=IxQtK2SjWWM#t=6.055047">vid</a></p>
<p><a href="http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture10.pdf">Stanford CS231n lecture course notes</a></p>
<p><a href="https://arxiv.org/pdf/1502.04623.pdf">Hard Attention
DRAW: A Recurrent Neural Network For Image Generation
Alex Graves et al.</a></p>
<p><a href="https://arxiv.org/pdf/1506.02025.pdf">Spatial Transformer Networks
Max Jaderberg Karen Simonyan Andrew Zisserman Koray Kavukcuoglu</a></p>
<p><a href="https://arxiv.org/pdf/1511.03416.pdf">Visual7W: Grounded Question Answering in Images
Yuke Zhu Oliver Groth Michael Bernstein Li Fei-Fei</a></p>
<p><a href="https://arxiv.org/pdf/1511.04119.pdf">Soft Attention
Action recognition using visual attention
Shikhar Sharma, Ryan Kiros & Ruslan Salakhutdinov</a></p>
<p><a href="https://arxiv.org/pdf/1604.03227.pdf">Recurrent Attentional Networks for Saliency Detection
Jason Kuen, Zhenhua Wang, Gang Wang</a></p>
<p><a href="https://arxiv.org/pdf/1511.03339.pdf">Attention to Scale: Scale-aware Semantic Image Segmentation
Liang-Chieh Chen et al.</a></p>
<p><a href="https://arxiv.org/pdf/1603.03925.pdf">Image Captioning with Semantic Attention
Quanzeng You at al.</a></p>
</div>
<div class="tag-cloud">
<p>
</p>
</div>
<div class="addthis_relatedposts_inline">
</article>
<footer>
<p>© </p>
<p> Powered by <a href="http://getpelican.com" target="_blank">Pelican</a> - <a href="https://github.com/alexandrevicenzi/flex" target="_blank">Flex</a> theme by <a href="http://alexandrevicenzi.com" target="_blank">Alexandre Vicenzi</a>
</p> </footer>
</main>
<script type="application/ld+json">
{
"@context" : "http://schema.org",
"@type" : "Blog",
"name": " Perth Machine Learning Group ",
"url" : "http://pmlg.github.io",
"image": "http://pmlg.github.io/images/network.png",
"description": "We are a group of students, industry professionals, entrepreneurs and curious individuals studying machine learning. We run beginner and advanced sessions every Thursday from 6-8pm at Flux, 191 St George's Terrace."
}
</script>
</body>
</html>