How to Speed Up Development on Larger Teams or Projects

Together we’ll go through some tools and cover how they can all work together to make your workflow easier no matter your team or project size. These tools will help you from your IDE all the way to deploying to production.

Overview:

IDE:

• A web based IDE so you can develop anywhere.

Scale Git:

• Scalar, to make it easy to work on the pieces of a git project that are relevant to you no matter the size of the repo.

Remote Build Execution with Caching and Multi Platform Support:

• Bazel, makes it fast to build and test large projects over hundreds or thousands of machines. Taking build times down from hours to minutes or minutes to seconds for all your devs and your ci/cd pipelines as well.

Continuous Integration:

• Prow, makes it easy to ensure all pull requests meet consistent standards across your team or organisation and automatically merge when these standards are met. For example ensure all tests pass, that prs are reviewed by at least one person and are automatically merged when ready.
• Tekton, easily create pipelines as code for your team or to share across teams or organisations. This helps elevate your pipelines from bash scripts or other home grown solutions and allows them to visible and improved by everyone.

Continuous Deployment:

• Spinnaker, quickly and easily get visibility into the current state of deployments across your team or organisation with one click rollbacks, many deployment techniques and pipelines as code. Helping you to see exactly whats happening and fix it quickly and easily.

How This All Fits Together

We’ll go through a workflow and how all of these tools fit together to make your development experience better and easier for everyone.

IDE

Now that everyone is working from home and people are enjoying the freedom that it provides this trend doesn’t seem like it’s going away anytime soon so why not use it to your advantage. Have a web based IDE that gives you a consistent work environment no matter the device or location your are working from that day. This also makes it easy to ensure updates and security patches are consistently applied to all member of the team. It also means there is nearly zero setup time for new people joining the team or those wanting to update or fix a blocking problem from another team. The reduced friction can improve collaboration and decrease the number of issues that are just thrown at a different team because the person who found it doesn’t have the environment setup to fix it. It can also help keep network speeds consistent for all the things you are doing in the IDE, this is useful if you are uploading or downloading things from your work environment. You won’t have to worry as much about the network speed in your home or wherever you’re working as long as you can connect to the IDE you are good!

Scale Git

As projects get bigger and their dependencies grow sometimes you don’t actually need everything in the repository to do your work for that task. You just need a subsection of the repository to build and test what you are working on. Scalar allows you to do this efficiently without having to memorise even more git commands. This means your front end and back end teams could work in the same repository with a common protocol buffer folder that each team has to pull down with either the front end or back end code. Allowing each team to work mostly independently while ensuring breaking changes are surfaced before they are merged into the main working branch.

Remote Build Execution with Caching and Multi Platform Support

Reduce your build times by up to 10x or more across all your developers and your continuous integration pipelines. Bazel’s remote build execution and caching allows builds to be distributed across hundreds or thousands or machines. This means instead of paying for a small vm that takes a long time to build or a large vm that costs a lot you can get all the compute power you could possibly need right away and only pay for what you use. This gives you the best of both worlds where you only pay for what you need and you get your builds done as fast as possible. Pairing this with a remote cache also means that you do not have to rebuild everything each time but just the pieces of the project that changed and the parts of the project that rely on those changed areas. For example if you are building a back end service and you make a change to section that no other piece of code relies on only that section will get rebuilt everything else with just be pulled from the cache saving time can compute costs.

Bazel is also working towards multi platform support where you could request that your build is executed on linux, windows and/or mac making it much easier to build for almost any platform from any platform. Work in this area is just beginning but points to a bright future for how bazel will make development even easier.

Continuous Integration

Prow makes it easier for your developers to get their changes merged in quickly with sustained standards and checks. If you submit changes and they pass all the build and test checks along with anything else your require and a review your changes will automatically be merged in as soon as possible. This means developers don’t have to wait for reviews then merge their changes in after. If everything looks good the changes will just be merged and everything continues moving forward. This system can be applied and set rules across your team or organisation and can scale with you as you grow.

The idea for this is to remove as many unnecessary steps between the developer creating code and getting it to production. You still want all the critical steps but non of the time consuming unhelpful steps and confirming the merge after it has passed all the tests and builds with an approving review is not required for most merges.

If you pair this with tekton you can create pipelines that are run for pre and post submit steps. These pipelines can be shared between teams and organisations making it easy to ensure everyone meets the same standards and also allows the overhead for each team to be reduced.

Continuous Deployment

Spinnaker gives you all the features you need to visualise and manage all your deployment pipelines as your changes are rolled out to production. This is a critical piece of your infrastructure and it should help you quickly understand any problems. While giving you the options to resolve those issues quickly to reduce any impact on your customers. Spinnaker can be used for small simple deployments and grow into multi stage complex deployments with different deployment schemes as needed. This gives your teams and organisations room to grow and add features as your product requires. The critical initial features are one-click rollbacks, automatic rollbacks, pipelines as code and many different deployment techniques built in and extensibility if more custom stages are needed.

Summary

All of these tools allow developers to be more efficient and increase velocity while at the same ensuring there are consistent checks and standards upheld across the code bases. Most importantly each step can catch and help fix errors that could have come through the previous steps making it easy and fast to fix errors along the way but still doing everything to ensure they don’t happen in the first place. This is key because believing that errors won’t get through because of one stage in your process or you have one tool along the way, almost always fails at some point increasing everyone's stress. But if each tool along the way can help mitigate problems or errors and has the features to minimise the impact of any errors that do get through. Everyone can feel better because problems can be found and resolved quickly and easily throughout the process. Making it much less stressful to deploy changes and in the end the stress of breaking something or a feature not working is usually the things that slows down deployments the most. If the stress is removed and everyone can feel confident that they did their best to make sure there were no problems even if something comes up the tools and processes in place can help mitigate and resolve problems quickly and efficiently. That really is the piece of mind all these tools are working to provide, an easy fast development process that can make every contributor feel safe and confident even in the event that a problem happens. Giving new or experienced contributors the same level of confidence and safety in their development process.

For more information email us at info@cosmodev.tech