I still prefer this version of the meme (which one day I will totally print it and put it on the back of my work laptop)
I’m almost a year in to a job where I was given this task with no admin access on my local windows machine, with a team that had never used an IDE or git before, and with only Google Drive as my allowed cloud tool. When I got here everything was just a bunch of Jupyter notebooks that would get run in Google Collab that were stored haphazardly over a shared Google Drive.
It’s been a slog, but Python for Windows, VSCode, Git for Windows, and Poetry can all be installed without admin access, and we got limited access to Azure DevOps. I’ve taught my team how to use powershell, git, VSCode, and Poetry, and taught them about testing and documentation (this is a slowwww process). We finally got a desktop computer with admin access this week that we can RDP into (that I requested basically right when I started), so we can run scheduled tasks on Windows and hack together some kind of a CI/CD system. We started a wiki on Azure, have most of our stuff documented and in a well organized monorepo, and track our work in boards now.
Now that other teams are starting to see how we’re doing things, they want in, too. Thank god these people are wonderful and excited to learn because otherwise this would be very frustrating.
Holy shit that must be equally frustrating and satisfying all at once - what a turn around!
Im an actuarie but everything I do is kn python jupyter notebooks,and I would like to do keep using them and use some git/version control with them. Is any good way to do that? Or are jupyter notebooks not git friendly?
Jupyter notebooks can totally handled by git! If you use GitHub, it will even render them on the WebUI for you.
I’m my past job we had Azure-devOps, i tried to upload an jupyter notebook but it didn’t recognized it was a jupyter notebook and show the file as a JSON and it was not nice to work with, I had to export the notebooks as python scripts to get it working fine. In my new job, I’ll still waiting for the IT team to approve and set up something for me.
Don’t wait. Come talk to us. Yeah things are hectic with demands flooding in from all directions but we want to make your job easier and better
The ticket is already open and I guess on the queue, and I already have a couple of more important tickets at front (some databases I want to access directly from python, instead of having to use excel to generate the queries and the export from it).
I don’t know if DevOps can render them. It certainly can’t on my system. I would recommend not using the remote repository WebUI for that feature.
In this new job I’m also looking up for the devops access (they even have github completely blocked on the corporate network) and I’m hoping I can connect it somehow with VS Code (in the pass one I couldn’t)
With jupyter notebooks in a devops perspective you could just build a process to export the notebooks to standard py files and then run them.
There are actually a lot of git hooks that will actually expoet/convert .ipynb to .py files automatically since notebooks don’t work great with git.
VS code can export and import from a to jupyter notebooks, but there’s some kind of bug and the imported notebooks always keep a ## % on each cell (not a high deal, but is annoying because subsequent exports/imports think they are cells to be created)
Our “big data” is
ActiveSheet.Rows.Countbig.ActiveSheet? Please noIt’s bigger than it looks: our data is spread across 64,000 worksheets.
That’s what MS Teams code blocks and ChatOps are for.
