Hmph.  I found this:

    Windows Forms controls do not support true transparency. The
    background of a transparent Windows Forms control is painted
    by its parent.

    (http://msdn2.microsoft.com/en-us/library/wk5b13s4.aspx)

I was able to reproduce the behavior you're reporting, and I admit it
surprised me.  But then I found the above quote, which suggests to me
that there's no straight-forward way to do what you want.

I tried some things involving creating a PictureBox-derived class.  I
thought maybe I could set the AllPaintingInWmPaint style and not doing
anything in the OnPaintBackground method, but in spite of the
implication in the docs that you can set the style and avoid having to
paint the background, Windows apparently expects that you will
completely draw the entire control when this style is set.  Simply not
drawing the part of the control that's transparent results in garbage,
presumably left over from temporary caching of drawing of the other
controls in the form (at least that's what it looks like to me).

Ordinarily, I would suggest skipping making the thing a control and just
drawing it directly yourself into the form.  However, it sounds as
though you want it to be drawn on top of other controls, and of course
all of the form's own rendering happens under any controls that are in
the form.  So that wouldn't be appropriate in your case.

If you can come up with a way to represent the transparency as a Region
(or rather, the area that's not transparent), then you could use that
Region as the Control's region, which would cause all drawing, including
the background, to be clipped to the region.  But if you're using a
plain bitmap, that could be problematic, as I'm not aware of any
built-in functions that would do that.  I suppose you could do it
yourself by scanning the bitmap line by line, checking the alpha of each
pixel, generating a Region that way (either pixel-by-pixel, or
consolidating adjacent non-transparent pixels in each line into a single
rectangle you add to the Region).  But that seems like a lot of work to
me.  :)

Unfortunately, that's all I have with respect to decent ideas.  So,
here's one last bad one: write your own picture-box control that copies
whatever's underneath it for use in drawing it's own background.  I
tried it, and it does work.  There's a little bit of "smudging" that
goes on as the control is moved, because some how it gets drawn in its
new location before the OnLocationChanged method is called.  This might
be fixable, but I think I'll leave that up to you to explore, at least
for now.  :)

I think this is a really hacky solution, and I'd probably try to use
something a little more "polite".  Also, this has potential performance
issues, since each time the control is moved, the entire form has to be
redrawn into a bitmap.  It's plenty fast on my fast PC with a small form
though.

Anyway, see if this is helpful at all (take this code and use it to
implement a custom control class in your own project):

using System;
using System.Drawing;
using System.Windows.Forms;

namespace TestTransparentGIF
{
    public partial class TransparentPictureBox : PictureBox
    {
        Bitmap _bmpBackground;
        bool _fDontDraw;

        public TransparentPictureBox()
        {
            InitializeComponent();

            SetStyle(ControlStyles.Opaque, true);
        }

        protected override void OnPaint(PaintEventArgs pe)
        {
            if (!_fDontDraw)
            {
                if (_bmpBackground == null)
                {
                    using (Bitmap bmpT = new Bitmap(Parent.Size.Width,
Parent.Size.Height))
                    {
                        Point ptParentScreen = Parent.Location;
                        Point ptScreen = Parent.PointToScreen(Location);
                        Point ptInParent;

                        if (Parent.Parent != null)
                        {
                            ptParentScreen =
Parent.Parent.PointToScreen(ptParentScreen);
                        }

                        ptInParent = ptScreen - new Size(ptParentScreen);

                        _fDontDraw = true;
                        Parent.DrawToBitmap(bmpT, new Rectangle(new
Point(), Parent.Size));
                        _fDontDraw = false;

                        _bmpBackground = new Bitmap(Width, Height);
                        using (Graphics gfxT =
Graphics.FromImage(_bmpBackground))
                        {
                            gfxT.DrawImageUnscaled(bmpT, new Point() -
new Size(ptInParent));
                        }
                    }
                }

                if (_bmpBackground != null)
                {
                    pe.Graphics.DrawImage(_bmpBackground, new Point());
                }

                // Calling the base class OnPaint
                base.OnPaint(pe);
            }
        }

        protected override void OnLocationChanged(EventArgs e)
        {
            if (_bmpBackground != null)
            {
                _bmpBackground.Dispose();
                _bmpBackground = null;

                Invalidate();
            }

            base.OnLocationChanged(e);
        }
    }
}