About lossy:
A digital image is like a mosaic of small units called pixels. Each of this pixels has a specific place, and also a specific colour. So your puter remembers its place, and its values in Red, Green and Blue. If you're working in RGB mode that is.
A bit of math: an image of 300 pixels wide and 200 pixels high contains 300x200=60.000pixels. Each pixels has three values: an R, a G and a B value, so your file will be some 60.000x3=180.000 bytes large. This is approximately 180KB.
If you save it as a psd (photoshop's native) or a tiff, it will be this size.
This is a lot for such a small image, so people developed several ways to make the file smaller. One is to zip it. Another one is to use the GIF format. Gif compresses, but is limited to a set of 256 colours. PNG's newer version isn't, but it is still rather large in filesize. That's why it still isn't popular.
So, for photographers, and people placing photographic images or paintings on their website, another solution had to be found. This is called JPG or jaypeg. What JPG does, is simplify your image, based on an observation that people don't always see the difference. Pixels that resemble one another, like in the clouds of a sky, are grouped into small units of the same RG and B values. This makes the filesize smaller as instead of having to remember each pixel, the puter memory can say: "that group, those values". As there are less groups, there is less information to be remembered, and the filesize is smaller. It even allows you to set the quality, meaning how accurate, or less accurate the pixels have to resemble one-another to become part of the group. You'll notice that even at maximum quality, the filesize is smaller.
When there are sharp transitions, like a sharp outline of a text, this method is less effective and the grouping beciomes visible as artefacts. This is a disadvantage.
But the biggest disadvantage is this: once the pixels of your image have been grouped by the jpg method, their individuality is lost and cannot be recuperated. You have a smaller filesize, but at the expense of losing information and quality. That's why jpg is called "lossy".
So if ever you have to work on a jpg, start by using the Save As option to change it into a psd or tiff. This way, when saving it you won't loose more info than necessary. If you have to save it again as jpg, then the file will undergo the same procedure, and you will loose even more information/quality as more pixels will be considered as being close enough in resemblance to be considers as having one and the same value.