Pantheon Community

CI services for the novice developer

I’m hoping that @mcdwayne is out there and listening. :slight_smile:

So… I can officially say that I am not afraid of bash and the command line in general. I am enjoying some of the powers that writing simple scripts can unlock. Thanks in no small part to the examples within this community, I’m safely out of the business of:

  • Look at the organization dashboard
  • Open ten sites in new tabs across my browser.
  • Click the update button in DEV for site 1.
  • Cycle through the other 9 sites for DEV
  • Repeat for Test. Visually inspect stuff.
  • Push to Live.
  • Repeat for the other 190+ sites in our organization.
  • Look up and realize that I lost a week to this process and I never want to the Pantheon dashboard screen again.

Now my problem is this one: I no longer want to leave my terminal running overnight with my script parked on my local machine. :nerd_face: No joke; I seriously have adopted a second old laptop which runs my terminal update scripts while I hack away on my day-to-day machine. While this works pretty well… it is completely inelegant. And at some point, I’m going to get some sand kicked in my face by one of those full-stack dev bullies on the playground.

So, time to swallow my pride and ask for some help. @mcdwayne’s recent article and WP talks mention three services that can provide me a “virtual laptop” to run my things in daylight hours. I’m ruling one of them out just because of inconvenience. (My code’s parked in GitHub and changing an upstream repo from one service to another is HARD.)

That leaves CircleCI and Jenkins. :robot:

My questions to you all are these: For the completely uninitiated developer, which of these tools is the easiest to pick up and run with? Any advice for someone taking the next steps in that direction?

Article reference:

I’ve seen Circle used more often than Jenkins. Circle’s SSH support into running containers makes it nice to debug. Not sure offhand what Circle’s timeout limit is though

If you only need a small server to run a bash script, you could look at the 3 big cloud providers. They give you the first year for free, then it’s a fairly small monthly fee after that.

https://aws.amazon.com/free/


Would take some minor setup, but then you can schedule a cron to run your bash script at any time and as often as you would like.

Woah!
Thank you so much for the kind words @SteveRyan-ASU It is humbling and ultra encouraging to hear you are able to leverage Bash more and more. I am fascinated by it and keep learning more by the day. I am actually just diving into Vim as an extension of the love. :slight_smile:

For your questions, All I know about CI in general I learned from @ataylorme and @greg and can recommend looking into their repositories.
I have never done the Amazon direct route but am actually going to dig into that for learning @Dan Thank you!
The More You Know! shooting star gif

@SteveRyan-ASU to @rvtraveller’s point of timeout limits a way around that is to create one job and run it multiple times. Some runs may sit in the queue for others to finish but having a job one once per site, instead of looping through 190+ sites in a single script, makes it easy to see when a specific site fails and keeps the job time down, avoiding timeouts.

The best way I’ve found to run the same job over and over for multiple sites is to use CircleCI’s feature that allows you to start jobs via the API. Essentially you run one job on a cron and all that job does is call terminus org:site:list (or similar) and fire off the job doing the work for each site in the list via API.

You can see an example of this in pantheon-training-org/build-tools-project-create repository. The create-sites.sh script is run (in my case locally but it could be on a cron) and all it does is start CircleCI jobs.

Doing it this way also allows scalability. If it takes too long to crawl through all 190+ sites by having each site run in its own job you can use more workers and run the jobs in parallel. I’m not trying to make a CircleCI sales pitch but if you need to speed up CI in general making your jobs smaller and running them concurrently is the way to go.

Thanks everyone for the responses. I’m going to clear my calendar in the near future to go learn some of this stuff, likely starting with CircleCI.

I appreciate the advice, assistance and good vibes. Oh, and the code… brilliant and amazing and full of the magic of the web. :star: Here’s to the adventure. :slight_smile: