PyCon 2024 showcased a number of ways to speed the pokey Python programming language including sub-interpreters, immortal objects, just-in-time compilation and more.
That’s why you need an architect to design the project for the expected requirements. They’ll ask the important questions, like:
how many users (if it’s a server)
any long-running processes? If so, what will those be doing?
how responsive does it need to be? What’s “fast enough”?
what’s more important short term, feature delivery or performance? Long term? How far away is “long term”?
what platforms do we need to support?
is AI or similar in the medium term goals? What’s the use case?
how bad is downtime? How much are you willing to spend if downtime isn’t an option?
You don’t need all the answers up front, but you need enough to design a coherent system. It’s like building a rail system, building a commuter line is much different than a light rail network, and the planners will need to know if those systems need to interact with anything else.
If you don’t do that, you’re going to end up overspending in some area, and probably significantly.
Profiling is an extremely useful tool for optimising the system that you have. It doesn’t help if you have the wrong system entirely though.
That’s why you need an architect to design the project for the expected requirements. They’ll ask the important questions, like:
You don’t need all the answers up front, but you need enough to design a coherent system. It’s like building a rail system, building a commuter line is much different than a light rail network, and the planners will need to know if those systems need to interact with anything else.
If you don’t do that, you’re going to end up overspending in some area, and probably significantly.
Upfront analysis and design is very close to independent from the technology, particularly at the I/O level