What is malloc doing, anyway?

When getting into wasm when it was new and had clang support, I wrote my own libc and malloc, which was so a great learning all these low level functions and tricks (free lists for example), I also like that I can read wasm quite well, and tooling is good to just return a .wat and read it.

Tangential, but the complexity and over-head of the malloc abstraction is precisely what made me very intrigued by "stack-only" languages, such a recently published Goose. There is something very elegant and exciting about the idea of (almost) completely getting rid of it. Just as modern programming languages have developed good ergonomics that replaces a garbage collector, the same could hopefully be done to replace heap-allocated memory.

[delayed]

I found this one very well explained time ago, for those looking for more details: https://sploitfun.wordpress.com/2015/02/10/understanding-gli...

   One of  my pet  peeves is  when people,  trying to  make a
   point, use code  that doesn't compile.  I  mean, it's fine
   when you're sketching  out code on a white  board.  But on
   an  article you've  had  time to  prepare,  it's a  little
   annoying.

   Take for instance, this code fragment:

   .. static void *ptrs[1000];
   .. 
   .. int main(void) {
   ..     for (int i = 0; i < 1000; i++)
   ..         ptrs[i] = malloc(16);
   ..     }
   ..  }

   I'm pretty sure what they //meant// to write was:

   .. #include <stdlib.h>
   ..
   .. static void *ptrs[1000];
   .. 
   .. int main(void) {
   ..     for (int i = 0; i < 1000; i++)
   ..         ptrs[i] = malloc(16);
   ..     }
   ..     return( 0 );
   ..  }

   I  understand why  the  author produced  the  code on  top
   instead of the code at the bottom.  The code on the bottom
   includes  features extraneous  to  the conversation.   The
   include  directive  and  return statement  only  serve  to
   distract from the point the author is making.

   If you read far enough down,  you'll see he has links to a
   broken GitHub  repo presumably  with code  fragments.  (or
   maybe  GitHub itself  is  broken today  or  it decided  it
   didn't  like me  trying to  access the  repo from  an AT&T
   fiber line.)

   I'm sure when GitHub decides to behave and serve bits like
   it's supposed to, there will be compilable code so readers
   can understand  the author's intent within  the context it
   was intended to be viewed.

   Which is to say:

   1. It's probably  okay to have only code  fragments in the
      main part  of your  discussion.  Please provide  a note
      that  there is  code which  shows your  snippet in  the
      context  of a  compilable program.   And put  that note
      somewhere near the code fragment.

      Or... consider putting  a note at the  beginning of the
      article saying  something like "Compilable  versions of
      code  fragments  in this  article  are  available at  a
      GitHub  repository  referenced  at the  bottom  of  the
      page."

   2. Do  not depend on  GitHub to hold your  code fragments.
      You do not  know when it will be down  or which readers
      it will randomly decide to block.

   3.  I really  do  think  you should  include  more of  the
      "layout"  code.   When   reading  "The  Address  Space"
      section,   it  was   unclear  where   the  initialized,
      uninitialized and local came from.  I initially thought
      the  author was  using a  version of  C I  had not  yet
      encountered that defined these symbols.

   4.  Since  the code  uses  ``sbrk()``,  the author  should
      probably have  cautioned readers  the code  isn't POSIX
      compliant, but should compile  and run on Linux systems
      using gcc.   There's a  comment about  demonstrating it
      with  glibc, but  I think  that's not  a strong  enough
      warning since you can  compile programs with clang that
      are intended to link against glibc.
      
      I doubt there  are many QNX users out  there who expect
      random code from  the internet to run  on their systems
      without alteration, but it  seems good form to describe
      the environment you are targeting.

      Now I'm wondering if this is expected to work on Darwin
      as well as Linux.

   5.  It  might  be  a  decent  idea  to  order  the  SHOW()
      statements in  the code in "The  Address Space" section
      to follow  the order  of the diagram  at the  bottom of
      that section:

      .. SHOW( "local (stack)", &local );
      .. SHOW( "printf (libc)", printf );
      .. SHOW( "program break", sbrk(0) );
      .. SHOW( "malloc(16) (heap)", malloc(16) );
      .. SHOW( "uninitialised (bss)", &uninitialised);
      .. SHOW( "initialised (data)", &initialised );
      .. SHOW( "main(text)", main );

   6.  In  the  "Chunks  and  Headers"  section,  it  may  be
      interesting to mention what the flags in the allocation
      headers actually mean.

   7. glibc does late fill,  right?  I can't remember.  If it
      does,  it might  be  useful to  mention  in the  Demand
      Paging section since it affects the program logic.
   
   Now that I've yelled at the  cloud a bit, allow me to say,
   I enjoyed the article.  Things I liked:

   1. The author  used the exact strace command  they used to
      produce  the  output  discussed  in  the  text  of  the
      article.

   2. The graphics are quite nice.

   3. Despite grousing about  fragments being fragments, they
      do a good job of illustrating points made in the text.

   EDIT: After complaining about  GitHub not being available,
   it's finally come back.  The web interface is still borked
   for me, but  I can clone the repo mentioned  at the bottom
   of the page and view the local copy of the source.

like many C functions you must remember that C is not low level, you won’t learn how memory is allocated by inspecting malloc() anymore than you would learn how characters are printed to the screen by reading printf() or how a file is written by reading write().

you will only learn about the thin or heavy wrapper that C is responsible for, the real magic is happening in the kernel and the device firmware

[flagged]

If the DRAM scarcity continues, not much.