Quite a few of us mentioned "glue" in our supply closets, and in particular, a glue gun. However, having the "right glue" for your materials is essential to its success and your sanity, and not even the "go-to-glue-gun" is always the right choice.  

I often find myself wondering "what's the RIGHT glue?"  ...as well as, wondering, "will it dry in time."  This is a particular problem for art projects that aren't just traditional "paper" crafts.

Here's a handy chart I found at http://www.u-createcrafts.com/craft-adhesive-guide/ that might also be "indispensable" in a supply closet:

Notes:  "E6000" is Michael's craft store's "glues to everything" brand. 

I also like to have types of CRAFT WIRE in my supply closet for those projects where glue just won't do.  (I learned the "wire it instead of glue" trick the hard way.)  Various sizes and colors of "beading" wire are sold in spools at craft stores. Great to have handy and also helps tie things together in a pinch (like backdrops. props and costumes), or to display a project.