logoalt Hacker News

robertlagrantlast Friday at 9:54 AM1 replyview on HN

> container images get fat at the python package installation layer, and that layer is one of the most frequently changed layers

This might be mitigated by having a standard set of packages, which you install in a lower layer, and then changing ones, at a higher layer.


Replies

elmolino89last Friday at 9:30 PM

Well, call them lazy but once you have i.e. biocontainers in which individual bioinformatics programs are prepackaged, hardly any scientist in that field would be reinventing the wheel and often just waste te time trying to install all the requirements and compile a program already running "good enough" using downloaded SIF. Sure, at times with say limited resources one can try to speed up some frequently used software creating SIF from scratch with say newer or more optimized Linux distro (if memory serves me right containers using Alpine Linux/musl library were a bit slower than containers using Ubuntu). But in the end splitting the input into smaller chunks, running i.e genome mapping on multiple nodes then combining the results, should be way faster than "turbo-charging" the genome mapping program run on a single node even with a big number of cores.