Hi all, As some of you know, I've been working for... quite some time now to try to secure funding for NumPy. So I'm excited that I can now officially announce that BIDS [1] is planning to hire several folks specifically to work on NumPy. These will full time positions at UC Berkeley, postdoc or staff, with probably 2 year (initial) contracts, and the general goal will be to work on some of the major priorities we identified at the last dev meeting: more flexible dtypes, better interoperation with other array libraries, paying down technical debt, and so forth. Though I'm sure the details will change as we start to dig into things and engage with the community. More details soon; universities move slowly, so nothing's going to happen immediately. But this is definitely happening and I wanted to get something out publicly before the conference season starts – so if you're someone who might be interested in coming to work with me and the other awesome folks at BIDS, then this is a heads-up: drop me a line and we can chat! I'll be at PyCon next week if anyone happens to be there. And feel free to spread the word. -n [1] http://bids.berkeley.edu/ -- Nathaniel J. Smith -- https://vorpus.org
Awesome! This is really great news. Does this mean is several person-years of funding secured? -CHB
On May 13, 2017, at 10:47 PM, Nathaniel Smith <njs@pobox.com> wrote:
Hi all,
As some of you know, I've been working for... quite some time now to try to secure funding for NumPy. So I'm excited that I can now officially announce that BIDS [1] is planning to hire several folks specifically to work on NumPy. These will full time positions at UC Berkeley, postdoc or staff, with probably 2 year (initial) contracts, and the general goal will be to work on some of the major priorities we identified at the last dev meeting: more flexible dtypes, better interoperation with other array libraries, paying down technical debt, and so forth. Though I'm sure the details will change as we start to dig into things and engage with the community.
More details soon; universities move slowly, so nothing's going to happen immediately. But this is definitely happening and I wanted to get something out publicly before the conference season starts – so if you're someone who might be interested in coming to work with me and the other awesome folks at BIDS, then this is a heads-up: drop me a line and we can chat! I'll be at PyCon next week if anyone happens to be there. And feel free to spread the word.
-n
-- Nathaniel J. Smith -- https://vorpus.org _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
On Sun, May 14, 2017 at 2:11 PM, Chris Barker - NOAA Federal <chris.barker@noaa.gov> wrote:
Awesome! This is really great news.
Does this mean is several person-years of funding secured?
Yes – hoping to give more details there soon. (There's nothing dire and secretive, it's just the logistics of getting an announcement approved by funder communication people didn't work with getting something out by PyCon, so this is the slightly confusing compromise.) -n -- Nathaniel J. Smith -- https://vorpus.org
Okay, a few more details :-) The initial funding here is a grant from the Gordon and Betty Moore Foundation to UCB with me as PI, in the amount of $645,020. There's also another thing in the pipeline that might supplement that, but it'll be ~6 months yet before we know for sure. So keep your fingers crossed I guess. Here's some text from the proposal (the references to "this year" may give some sense of how long this has taken...): https://docs.google.com/document/d/1xHjQqc8V8zJk7WSCyw9NPCpMYZ2Urh0cmFm2vDd1... -n On Sun, May 14, 2017 at 3:40 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Sun, May 14, 2017 at 2:11 PM, Chris Barker - NOAA Federal <chris.barker@noaa.gov> wrote:
Awesome! This is really great news.
Does this mean is several person-years of funding secured?
Yes – hoping to give more details there soon. (There's nothing dire and secretive, it's just the logistics of getting an announcement approved by funder communication people didn't work with getting something out by PyCon, so this is the slightly confusing compromise.)
-n
-- Nathaniel J. Smith -- https://vorpus.org
-- Nathaniel J. Smith -- https://vorpus.org
On Sat, May 13, 2017 at 11:45 PM, Nathaniel Smith <njs@pobox.com> wrote:
Hi all,
As some of you know, I've been working for... quite some time now to try to secure funding for NumPy. So I'm excited that I can now officially announce that BIDS [1] is planning to hire several folks specifically to work on NumPy. These will full time positions at UC Berkeley, postdoc or staff, with probably 2 year (initial) contracts, and the general goal will be to work on some of the major priorities we identified at the last dev meeting: more flexible dtypes, better interoperation with other array libraries, paying down technical debt, and so forth. Though I'm sure the details will change as we start to dig into things and engage with the community.
More details soon; universities move slowly, so nothing's going to happen immediately. But this is definitely happening and I wanted to get something out publicly before the conference season starts – so if you're someone who might be interested in coming to work with me and the other awesome folks at BIDS, then this is a heads-up: drop me a line and we can chat! I'll be at PyCon next week if anyone happens to be there. And feel free to spread the word.
Excellent news. Do you have any sort of timeline in mind? It will be interesting to see what changes this leads to, both in the code and in the project sociology. Chuck
On Sun, May 14, 2017 at 2:56 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Sat, May 13, 2017 at 11:45 PM, Nathaniel Smith <njs@pobox.com> wrote:
Hi all,
As some of you know, I've been working for... quite some time now to try to secure funding for NumPy. So I'm excited that I can now officially announce that BIDS [1] is planning to hire several folks specifically to work on NumPy. These will full time positions at UC Berkeley, postdoc or staff, with probably 2 year (initial) contracts, and the general goal will be to work on some of the major priorities we identified at the last dev meeting: more flexible dtypes, better interoperation with other array libraries, paying down technical debt, and so forth. Though I'm sure the details will change as we start to dig into things and engage with the community.
More details soon; universities move slowly, so nothing's going to happen immediately. But this is definitely happening and I wanted to get something out publicly before the conference season starts – so if you're someone who might be interested in coming to work with me and the other awesome folks at BIDS, then this is a heads-up: drop me a line and we can chat! I'll be at PyCon next week if anyone happens to be there. And feel free to spread the word.
Excellent news. Do you have any sort of timeline in mind?
The exact timeline's going to be determined in large part by university+funder logistics. I thought it was going to happen last year, so at this point I'm just going with the flow :-). The process for hiring staff definitely takes a few months at a minimum; with postdocs there's a little more flexibility. -n -- Nathaniel J. Smith -- https://vorpus.org
Hi, On Sun, May 14, 2017 at 10:56 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Sat, May 13, 2017 at 11:45 PM, Nathaniel Smith <njs@pobox.com> wrote:
Hi all,
As some of you know, I've been working for... quite some time now to try to secure funding for NumPy. So I'm excited that I can now officially announce that BIDS [1] is planning to hire several folks specifically to work on NumPy. These will full time positions at UC Berkeley, postdoc or staff, with probably 2 year (initial) contracts, and the general goal will be to work on some of the major priorities we identified at the last dev meeting: more flexible dtypes, better interoperation with other array libraries, paying down technical debt, and so forth. Though I'm sure the details will change as we start to dig into things and engage with the community.
More details soon; universities move slowly, so nothing's going to happen immediately. But this is definitely happening and I wanted to get something out publicly before the conference season starts – so if you're someone who might be interested in coming to work with me and the other awesome folks at BIDS, then this is a heads-up: drop me a line and we can chat! I'll be at PyCon next week if anyone happens to be there. And feel free to spread the word.
Excellent news. Do you have any sort of timeline in mind?
It will be interesting to see what changes this leads to, both in the code and in the project sociology.
I was thinking the same thing - if this does come about, it would likely have a big impact on practical governance. It could also mean that more important development conversations happen off-list. It seems to me it would be good to plan for this consciously. Cheers, Matthew
Great news, Nathaniel! It was a huge boost to matplotlib a couple of years ago when we got an FTE, even if it was just for a few months. While that effort didn't directly produce any new features, we were able to overhaul some very old parts of the codebase. Probably why the effort was so successful was that, 1) Michael had a clear idea of what needed work and how to achieve it and 2) the components impacted were mostly not user-facing. With respect to off-list conversations, one thing that the matplotlib devs have done is set up a weekly Google Hangouts session. A summary of that meeting is then posted to the mailing list. A practice like that (posting summaries of regular meetings) might be sufficient to feed off-line discussions back to the greater community. Cheers! Ben Root On Mon, May 15, 2017 at 4:43 AM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Sun, May 14, 2017 at 10:56 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Sat, May 13, 2017 at 11:45 PM, Nathaniel Smith <njs@pobox.com> wrote:
Hi all,
As some of you know, I've been working for... quite some time now to try to secure funding for NumPy. So I'm excited that I can now officially announce that BIDS [1] is planning to hire several folks specifically to work on NumPy. These will full time positions at UC Berkeley, postdoc or staff, with probably 2 year (initial) contracts, and the general goal will be to work on some of the major priorities we identified at the last dev meeting: more flexible dtypes, better interoperation with other array libraries, paying down technical debt, and so forth. Though I'm sure the details will change as we start to dig into things and engage with the community.
More details soon; universities move slowly, so nothing's going to happen immediately. But this is definitely happening and I wanted to get something out publicly before the conference season starts – so if you're someone who might be interested in coming to work with me and the other awesome folks at BIDS, then this is a heads-up: drop me a line and we can chat! I'll be at PyCon next week if anyone happens to be there. And feel free to spread the word.
Excellent news. Do you have any sort of timeline in mind?
It will be interesting to see what changes this leads to, both in the
code
and in the project sociology.
I was thinking the same thing - if this does come about, it would likely have a big impact on practical governance. It could also mean that more important development conversations happen off-list. It seems to me it would be good to plan for this consciously.
Cheers,
Matthew _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
On Mon, May 15, 2017 at 1:43 AM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Sun, May 14, 2017 at 10:56 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Sat, May 13, 2017 at 11:45 PM, Nathaniel Smith <njs@pobox.com> wrote:
Hi all,
As some of you know, I've been working for... quite some time now to try to secure funding for NumPy. So I'm excited that I can now officially announce that BIDS [1] is planning to hire several folks specifically to work on NumPy. These will full time positions at UC Berkeley, postdoc or staff, with probably 2 year (initial) contracts, and the general goal will be to work on some of the major priorities we identified at the last dev meeting: more flexible dtypes, better interoperation with other array libraries, paying down technical debt, and so forth. Though I'm sure the details will change as we start to dig into things and engage with the community.
More details soon; universities move slowly, so nothing's going to happen immediately. But this is definitely happening and I wanted to get something out publicly before the conference season starts – so if you're someone who might be interested in coming to work with me and the other awesome folks at BIDS, then this is a heads-up: drop me a line and we can chat! I'll be at PyCon next week if anyone happens to be there. And feel free to spread the word.
Excellent news. Do you have any sort of timeline in mind?
It will be interesting to see what changes this leads to, both in the code and in the project sociology.
I was thinking the same thing - if this does come about, it would likely have a big impact on practical governance. It could also mean that more important development conversations happen off-list. It seems to me it would be good to plan for this consciously.
Yeah, definitely. Being able to handle changes like this was one of the major motivations for all the governance discussions we started a few years ago, and it's something we'll need to keep an eye on going forward. To state it explicitly though: the idea is to fund folks so that they can contribute to numpy within our existing process of open community review, and preserving and growing that community is very much one of the grant's goals; no-one should get special privileges because of where their paycheck is coming from. If at some point you (or anyone) feel like we're deviating from that please speak up. -n -- Nathaniel J. Smith -- https://vorpus.org
Hi, On Fri, May 19, 2017 at 10:06 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Mon, May 15, 2017 at 1:43 AM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Sun, May 14, 2017 at 10:56 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Sat, May 13, 2017 at 11:45 PM, Nathaniel Smith <njs@pobox.com> wrote:
Hi all,
As some of you know, I've been working for... quite some time now to try to secure funding for NumPy. So I'm excited that I can now officially announce that BIDS [1] is planning to hire several folks specifically to work on NumPy. These will full time positions at UC Berkeley, postdoc or staff, with probably 2 year (initial) contracts, and the general goal will be to work on some of the major priorities we identified at the last dev meeting: more flexible dtypes, better interoperation with other array libraries, paying down technical debt, and so forth. Though I'm sure the details will change as we start to dig into things and engage with the community.
More details soon; universities move slowly, so nothing's going to happen immediately. But this is definitely happening and I wanted to get something out publicly before the conference season starts – so if you're someone who might be interested in coming to work with me and the other awesome folks at BIDS, then this is a heads-up: drop me a line and we can chat! I'll be at PyCon next week if anyone happens to be there. And feel free to spread the word.
Excellent news. Do you have any sort of timeline in mind?
It will be interesting to see what changes this leads to, both in the code and in the project sociology.
I was thinking the same thing - if this does come about, it would likely have a big impact on practical governance. It could also mean that more important development conversations happen off-list. It seems to me it would be good to plan for this consciously.
Yeah, definitely. Being able to handle changes like this was one of the major motivations for all the governance discussions we started a few years ago, and it's something we'll need to keep an eye on going forward. To state it explicitly though: the idea is to fund folks so that they can contribute to numpy within our existing process of open community review, and preserving and growing that community is very much one of the grant's goals; no-one should get special privileges because of where their paycheck is coming from. If at some point you (or anyone) feel like we're deviating from that please speak up.
I think Chuck's term 'sociology' is a good one; although it's good to have the governance document, I don't think it covers all the issues that your initiative brings up. At the moment, the people doing the most work on numpy (by commits at least) are Chuck, Eric and Julian. As far as I know, we don't have any full-time developers. When you've got up and running it sounds like you'll have at least two full-time developers. I guess you'll be their manager, and that they will be physically housed in the BIDS. So, this will represent a big shift in practical influence, from a more or less completely distributed pool of developer hours, to something much closer to an institution-owned project. Of course, we can hope that this doesn't have any negative consequences in terms of project dynamics, but it seems to me that it would be sensible to think of the risks, and plan for them, rather than waiting for the expected problems to arise, when they may be too late to fix. We had this kind of discussion with Travis a while ago, and he pushed back about conflicts of interests for people in the BIDS. At the time, that wasn't a serious issue, because you were the only BIDS member actively committing to numpy. That changes when it's you, and two full-time developers. Cheers, Matthew
Hi All, First, it will be great to have more people developing! On avoiding potential conflicts: I'm not overly worried, in part because of my experience with astropy (for which NASA support developers at STScI and CXC). One possible solution for trying to avoid them would be to adapt the typical academic rule for conflicts of interests to PRs, that non-trivial ones cannot be merged by someone who has a conflict of interest with the author, i.e., it cannot be a superviser, someone from the same institute, etc. But probably the larger issue is how to communicate larger ideas about where to go, ideas which might form "over coffee" at BIDS. We have NEPs and this mailing list, but neither seem particularly effective, in part because it simply is hard to take enough time to really think through the more complicated ideas that would be the whole purpose of a NEP. I don't know that this has a solution beyond trying to ensure that one always involves people who should be interested in it. All the best, Marten
Hi, On Sun, May 21, 2017 at 7:27 PM, Marten van Kerkwijk <m.h.vankerkwijk@gmail.com> wrote:
Hi All,
First, it will be great to have more people developing! On avoiding potential conflicts: I'm not overly worried, in part because of my experience with astropy (for which NASA support developers at STScI and CXC). One possible solution for trying to avoid them would be to adapt the typical academic rule for conflicts of interests to PRs, that non-trivial ones cannot be merged by someone who has a conflict of interest with the author, i.e., it cannot be a superviser, someone from the same institute, etc.
There's surely a spectrum from "I'm sure it's going to be fine, let's just see what happens" to detailed documentation of procedure and management. In this case I'm arguing for something fairly well to the right of "I'm sure it's going to be fine" - it seems to me that we could get 80% of the way to a reassuring blueprint with a relatively small amount of effort. Cheers, Matthew
Hi Matthew,
it seems to me that we could get 80% of the way to a reassuring blueprint with a relatively small amount of effort.
My sentence "adapt the typical academic rule for conflicts of interests to PRs, that non-trivial ones cannot be merged by someone who has a conflict of interest with the author, i.e., it cannot be a superviser, someone from the same institute, etc." was meant as a suggestion for part of this blueprint! I'll readily admit, though, that since I'm not overly worried, I haven't even looked at the policies that are in place, nor do I intend to contribute much beyond this e-mail. Indeed, it may be that the old adage "every initiative is punishable" holds here... would you, or one of the others who feels it is important to have a blueprint, be willing to provide a concrete text for discussion? All the best, Marten
On Mon, May 22, 2017 at 11:52 AM, Marten van Kerkwijk < m.h.vankerkwijk@gmail.com> wrote:
My sentence "adapt the typical academic rule for conflicts of interests to PRs, that non-trivial ones cannot be merged by someone who has a conflict of interest with the author, i.e., it cannot be a superviser, someone from the same institute, etc." was meant as a suggestion for part of this blueprint!
This sounds like a good rule of thumb to me. As a practical matter, asking someone outside to approve changes is a good way to ensure that decisions are not short-circuited by offline discussions. But remember that per our governance procedures, we already require consensus for decision making. So I don't think we need an actual change here. I'll readily admit, though, that since I'm not overly worried, I
haven't even looked at the policies that are in place, nor do I intend to contribute much beyond this e-mail.
I am also not worried about this, really not at all. NumPy already has governance procedures and a steering committee for handling exactly these sorts of concerns, should they arise (which I also consider extremely unlikely in the case of BIDS and their non-profit funder).
Hi, On Mon, May 22, 2017 at 4:52 PM, Marten van Kerkwijk <m.h.vankerkwijk@gmail.com> wrote:
Hi Matthew,
it seems to me that we could get 80% of the way to a reassuring blueprint with a relatively small amount of effort.
My sentence "adapt the typical academic rule for conflicts of interests to PRs, that non-trivial ones cannot be merged by someone who has a conflict of interest with the author, i.e., it cannot be a superviser, someone from the same institute, etc." was meant as a suggestion for part of this blueprint!
I'll readily admit, though, that since I'm not overly worried, I haven't even looked at the policies that are in place, nor do I intend to contribute much beyond this e-mail. Indeed, it may be that the old adage "every initiative is punishable" holds here...
I understand what you're saying, but I think a more helpful way of thinking of it, is putting the groundwork in place for the most fruitful possible collaboration.
would you, or one of the others who feels it is important to have a blueprint, be willing to provide a concrete text for discussion?
It doesn't make sense for me to do that, I'm #13 for commits in the last year. I'm just one of the many people who completely depend on numpy. Also, taking a little time to think these things through seems like a small investment with the potential for significant gain, in terms of improving communication and mitigating risk. So, I think my suggestion is that it would be a good idea for Nathaniel and the current steering committee to talk through how this is going to play out, how the work will be selected and directed, and so on. Cheers, Matthew
On Mon, 2017-05-22 at 17:35 +0100, Matthew Brett wrote:
Hi,
On Mon, May 22, 2017 at 4:52 PM, Marten van Kerkwijk <m.h.vankerkwijk@gmail.com> wrote:
Hi Matthew,
it seems to me that we could get 80% of the way to a reassuring blueprint with a relatively small amount of effort.
My sentence "adapt the typical academic rule for conflicts of interests to PRs, that non-trivial ones cannot be merged by someone who has a conflict of interest with the author, i.e., it cannot be a superviser, someone from the same institute, etc." was meant as a suggestion for part of this blueprint!
I'll readily admit, though, that since I'm not overly worried, I haven't even looked at the policies that are in place, nor do I intend to contribute much beyond this e-mail. Indeed, it may be that the old adage "every initiative is punishable" holds here...
I understand what you're saying, but I think a more helpful way of thinking of it, is putting the groundwork in place for the most fruitful possible collaboration.
would you, or one of the others who feels it is important to have a blueprint, be willing to provide a concrete text for discussion?
It doesn't make sense for me to do that, I'm #13 for commits in the last year. I'm just one of the many people who completely depend on numpy. Also, taking a little time to think these things through seems like a small investment with the potential for significant gain, in terms of improving communication and mitigating risk.
So, I think my suggestion is that it would be a good idea for Nathaniel and the current steering committee to talk through how this is going to play out, how the work will be selected and directed, and so on.
Frankly, I would suggest to wait for now and ask whoever is going to get the job to work out how they think it should be handled. And then we complain if we expect more/better ;). For now I only would say that I will expect more community type of work then we now often manage to do. And things such as meticulously sticking to writing NEPs. So the only thing I can see that might be good is putting "community work" or something like it specifically as part of the job description, and thats up to Nathaniel probably. Some things like not merging large changes by two people sittings in the same office should be obvious (and even if it happens, we can revert). But its nothing much new there I think. - Sebastian
Cheers,
Matthew _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
On Mon, May 22, 2017 at 10:04 AM, Sebastian Berg <sebastian@sipsolutions.net> wrote:
On Mon, 2017-05-22 at 17:35 +0100, Matthew Brett wrote:
Hi,
On Mon, May 22, 2017 at 4:52 PM, Marten van Kerkwijk <m.h.vankerkwijk@gmail.com> wrote:
Hi Matthew,
it seems to me that we could get 80% of the way to a reassuring blueprint with a relatively small amount of effort.
My sentence "adapt the typical academic rule for conflicts of interests to PRs, that non-trivial ones cannot be merged by someone who has a conflict of interest with the author, i.e., it cannot be a superviser, someone from the same institute, etc." was meant as a suggestion for part of this blueprint!
I'll readily admit, though, that since I'm not overly worried, I haven't even looked at the policies that are in place, nor do I intend to contribute much beyond this e-mail. Indeed, it may be that the old adage "every initiative is punishable" holds here...
I understand what you're saying, but I think a more helpful way of thinking of it, is putting the groundwork in place for the most fruitful possible collaboration.
would you, or one of the others who feels it is important to have a blueprint, be willing to provide a concrete text for discussion?
It doesn't make sense for me to do that, I'm #13 for commits in the last year. I'm just one of the many people who completely depend on numpy. Also, taking a little time to think these things through seems like a small investment with the potential for significant gain, in terms of improving communication and mitigating risk.
So, I think my suggestion is that it would be a good idea for Nathaniel and the current steering committee to talk through how this is going to play out, how the work will be selected and directed, and so on.
Frankly, I would suggest to wait for now and ask whoever is going to get the job to work out how they think it should be handled. And then we complain if we expect more/better ;).
This is roughly where I am as well. Certainly this is an important issue, but we've already done a lot of groundwork in the abstract – the dev meeting, formalizing the governance document, and so forth (and recall that "let's get to a point where we can apply for grants" was explicitly one of the goals in those discussions). I think at this point the most productive thing to do is wait until we have a more concrete picture of who/what/when will be happening, so we can make a concrete plan.
For now I only would say that I will expect more community type of work then we now often manage to do. And things such as meticulously sticking to writing NEPs. So the only thing I can see that might be good is putting "community work" or something like it specifically as part of the job description,
Definitely.
and thats up to Nathaniel probably.
Some things like not merging large changes by two people sittings in the same office should be obvious (and even if it happens, we can revert). But its nothing much new there I think.
-n -- Nathaniel J. Smith -- https://vorpus.org
+1 for a conflict of interest policy. A member of another group reviewing and pulling a change is a reasonable expectation. Explicit is better than implicit.
participants (9)
-
Benjamin Root -
Charles R Harris -
Chris Barker - NOAA Federal -
Elliot Hallmark -
Marten van Kerkwijk -
Matthew Brett -
Nathaniel Smith -
Sebastian Berg -
Stephan Hoyer