What can I do to avoid air leaks from glass windows without replacing it fully?

Of course the glass doesn't leak. If you can feel cold air that seems to come through the glass, that is conduction. There are two ways to fix that: double wall panes, and styrofoam sheets. Insulating glass is rather costly. Styrofoam sheets are sold at lumber stores. You can cut them with a very sharp knife and they will fit into the window frame by friction. Use tape if you didn't cut accurately. Pieces can be edge glued with white glue (Elmer's) and tape the joints.

Cold air often comes in through cracks where the window frame meets the wall. You can close those with glue or tape. Glue is preferred. Cold air also comes in through switches and outlets on outside walls. Hardware stores sell sealers for those. Get new weatherstripping for all outside doors, and check the sweep at the bottom of the door too.