What is the factors that affect ionization energy of an atom is?

Everything about an atom is determined by electric charge. The proton is positive so it pulls an electron in close. Two protons don't stick together, so they need neutrons between them. They pull two electrons in, but the electrons repel each other, so they can't go just anywhere, but a helium atom is still very small. Then you go to the third atom and the electron has to go into a shell farther out, so it is farther from the positive nucleus. That makes it easy to remove, meaning low ionization potential. If you try to add an electron, the positive nucleus is kinda far away so it's not a very strong setup.

It's ionization potential, not energy. Potential means voltage and that means a tension, because it all depends on electric charge. Energy is not a physical unit. It is a philosophical concept and an accounting technique used to analyze mechanical and chemical exchanges. In nuclear physics it is defined as a wavelength of light, in biology it is a synonym for metabolism, and in public utilities they say energy when they mean connectivity. Anybody who uses the word in any other context either does not know what he is talking about, or is peddling something you don't need.