Proposal draft: Enhance Amy workshop management tool

[pbanaszkiewicz]

Ref #6

[rgaiacs]

@pbanaszkiewicz looks good to me.

[wking]

Maybe “is growing rapidly” instead of “dynamically grows”? I know Greg publishes charts of students-taught in the blog. I don't know if he publishes charts of workshops-held, but linking to either of those from here would be nice.

[pbanaszkiewicz]

I was unable to find numbers so I inquired @gvwilson about them.

[wking]

On Tue, Mar 24, 2015 at 11:38:20AM -0700, Piotr Banaszkiewicz wrote:

+The number of Software Carpentry's workshops run weekly
dynamically grows. All

I was unable to find numbers so I inquired @gvwilson about them.

So I'm not sure about workshops, but I think this is the last graph
showing the increase in student numbers 1.

[pbanaszkiewicz]

Oh nice, thank you.

I've just got data from Greg, I'm producing an updated chart. Not sure where to publish it, though.

[pbanaszkiewicz]

Nice :-)

[wking]

On Tue, Mar 24, 2015 at 01:32:00PM -0700, Piotr Banaszkiewicz wrote:

You need to sort your data to avoid the downward jiggle around April
2014.

[pbanaszkiewicz]

I'm not able to get rid of jiggling entirely - this may be because I have a dataset with non-unique index (aka date). Very small jiggling is visible even after sorting.

(To fix horrible jiggling in this one I removed entries with duplicated dates leaving the latest entry with biggest number of instructors)

Anyway, I'll include two nicest looking plots I could get to this proposal and later change them to links - if they get published. Otherwise I'll link to the plot you pointed out earlier (#36 (comment)).

[wking]

On Tue, Mar 24, 2015 at 02:21:07PM -0700, Piotr Banaszkiewicz wrote:

I'm not able to get rid of jiggling entirely - this may be because I
have a dataset with non-unique index (aka date).

Right, you want to sort by increasing count (which will automatically
sort by increasing date). It looks like you currently have multiple
entries for one date and the higher-count entry is currently landing
before the lower count entry.

To fix horrible jiggling in this one I removed entries with
duplicated dates leaving the latest entry with biggest number of
instructors

That works. I'm not sure how you're creating the plots, but most
plotting packages have something that lets you draw stepped lines.
You should use something like gnuplot's 'steps' 1. With that
plotting style, it won't matter whether you de-dup the dates (as long
as you've sorted by increasing count).

[pbanaszkiewicz]

I'm using Pandas with Seaborn for nicer colors.

Pandas is very straightforward when it comes to reading and plotting CSV date-indexed files:

df = pandas.read_csv("enrolment_workshops_data.csv", index_col=0, parse_dates=True)
df.plot()

However, if I understand correctly, Pandas has support for multi-index (multi-indices?) data frames. It would take me 5 minutes to remove duplicate data points from the CSV files, but way longer to figure out how to do that in Pandas.

[wking]

On Tue, Mar 24, 2015 at 02:55:38PM -0700, Piotr Banaszkiewicz wrote:

However, if I understand correctly, Pandas has support for
multi-index (multi-indices?) data frames. It would take me 5 minutes
to remove duplicate data points from the CSV files, but way longer
to figure out how to do that in Pandas.

Or you could load the CSV with the Python stdlib, sort and dedup, and
then pass that to Pandas ;). But still, it may not be worth the
trouble.

[wking]

I added a handful of copy-edit comments, but the meat of this proposal looks great to me.

[pbanaszkiewicz]

Thanks @wking for your comments and suggestions :) Now I only need to find some source to back up my "SwC is growing rapidly" claim.

[pbanaszkiewicz]

I added some plots to the proposal - you can view the rendered proposal with the plots here.

[wking]

On Tue, Mar 24, 2015 at 02:26:16PM -0700, Piotr Banaszkiewicz wrote:

https://github.com/pbanaszkiewicz/gsoc/blob/pbanaszkiewicz-proposal2/2015/proposals/banaszkiewicz-piotr-amy.md

Probably use a right-side y axis for workshops if you're plotting both
enrollment and workshop counts in the same figure. Otherwise the
workshop line is squashed way down at the bottom ;). And it looks
like this is currently showing “enrollment/workshops” and then
“workshops”. Maybe you want “enrollment/workshops” and then
“instructors”?

[pbanaszkiewicz]

Hey @wking,

Probably use a right-side y axis for workshops if you're plotting both enrollment and workshop counts in the same figure. Otherwise the workshop line is squashed way down at the bottom ;)

Indeed, the workshops line is squashed. I was thinking about a log scale, but then the plot would not be intuitive.

And it looks like this is currently showing “enrollment/workshops” and then “workshops”.

Yes, and that was intended. I wanted to first show how few workshops we run compared to number of people reached.

Then I wanted to show how workshops line really looks.

[wking]

On Tue, Mar 24, 2015 at 02:59:42PM -0700, Piotr Banaszkiewicz wrote:

And it looks like this is currently showing “enrollment/workshops” and then “workshops”.

Yes, and that was intended. I wanted to first show how few workshops
we run compared to number of people reached.

Then I wanted to show how workshops line really looks.

I'd spell that out in the surrounding text then.

[rgaiacs]

@pbanaszkiewicz Thanks for your proposal.

You need to submit your proposal to https://www.google-melange.com/gsoc/homepage/google/gsoc2015 as soon as possible. The deadline is March 27th 19:00 UTC.

[gvwilson]

Ship it!

[rgaiacs]

I'm merging this issue since student application period is over.