I found the problem.  I realized when using the toolbox the added item is
automatically added to the controls collection of the root component.  When
you programmatically add an item you need to add it both to the IContainer
and to the root component.Controls collection.

Even that does not solve the problem if you are trying to add a control to a Web page.
The following sample code works fine, but the designer does not show up the new control. However, if I try to drag & drop another control and change its ID property to match the one of my dynamic control, I get a designer error stating that a control with that ID already exists (although nobody can see it)

                       Dim host As IDesignerHost = CType(Me.Page.Site.GetService(GetType(IDesignerHost)), IDesignerHost)
                       Dim myTransaction As System.ComponentModel.Design.DesignerTransaction = host.CreateTransaction

                       MsgBox("Before adding the control there are " & host.Container.Components.Count.ToString & " controls on the designer")
                       MsgBox("Before adding the control there are " & Me.Page.Controls.Count.ToString & " controls on the page")
                       Dim newItem As IComponent = host.CreateComponent(GetType(System.Web.UI.WebControls.LinkButton))
                       Dim dcd As IDesigner = host.GetDesigner(newItem)
                       'dcd.Initialize(newItem)
                       Dim myNewLinkButton As System.Web.UI.WebControls.LinkButton = DirectCast(newItem, System.Web.UI.WebControls.LinkButton)
                       myNewLinkButton.Text = "This is a sample text"

                       myNewLinkButton.ID = "t" & Now.Millisecond.ToString
                       myNewLinkButton.Visible = True

                       dcd.Component.Site.Name = myNewLinkButton.ID
                       MsgBox(myNewLinkButton.ID)

                       'host.Container.Add(myNewLinkButton)
                       CType(host.RootComponent, System.Web.UI.Page).Controls.Add(CType(myNewLinkButton, System.Web.UI.WebControls.LinkButton))

                       Dim c As IComponentChangeService = DirectCast(host.RootComponent.Site.GetService(GetType(IComponentChangeService)), IComponentChangeService)

                       c.OnComponentChanging(myNewLinkButton, Nothing)
                       c.OnComponentChanging(host.RootComponent, Nothing)

                       c.OnComponentChanged(myNewLinkButton, Nothing, Nothing, Nothing)
                       c.OnComponentChanged(host.RootComponent, Nothing, Nothing, Nothing)

                       myTransaction.Commit()

                       MsgBox("After adding the controls there are " & host.Container.Components.Count.ToString & " controls on the designer")
                       MsgBox("After adding the control there are " & Me.Page.Controls.Count.ToString & " controls on the page")

                       Dim sel As ISelectionService = DirectCast(host.RootComponent.Site.GetService(GetType(ISelectionService)), ISelectionService)
                       sel.SetSelectedComponents(New IComponent() {myNewLinkButton})

From http://www.developmentnow.com/g/32_2004_9_0_0_123607/Adding-an-item-to-a-designe
r-host-programmatically.ht

I have solved the problem.
The following code sample will programmatically add a control to a ASPX Web Form from the INIT method of another control.
It solves the problem of the designer not displaying the control.
Enjoy!

'Get a Handle of the Designer
                       Dim host As IDesignerHost = CType(Me.Page.Site.GetService(GetType(IDesignerHost)), IDesignerHost)
                       Dim myTransaction As System.ComponentModel.Design.DesignerTransaction = host.CreateTransaction

                       MsgBox("Before adding the control there are " & host.Container.Components.Count.ToString & " controls on the designer")
                       MsgBox("Before adding the control there are " & Me.Page.Controls.Count.ToString & " controls on the page")

                       'Create a component
                       Dim newItem As IComponent = host.CreateComponent(GetType(System.Web.UI.WebControls.LinkButton))
                       Dim dcd As IDesigner = host.GetDesigner(newItem)
                       'This method of creating the component would also work
                       'Dim newItem2 As IComponent = CType(System.Activator.CreateInstance(GetType(System.Web.UI.WebControls.LinkButton)), System.Web.UI.WebControls.LinkButton)

                       'Set properties for the newly created control
                       Dim myNewLinkButton As System.Web.UI.WebControls.LinkButton = DirectCast(newItem, System.Web.UI.WebControls.LinkButton)
                       myNewLinkButton.Text = "This is a sample text"
                       myNewLinkButton.ID = "t" & Now.Millisecond.ToString 'Do not forget to assign an ID
                       dcd.Component.Site.Name = myNewLinkButton.ID 'If you do not do this, when you manually add a control of the same type you'd get an error

                       'host.Container.Add(myNewLinkButton) 'Only required if you had used Activator.CreateInstance to create the component
                       CType(host.RootComponent, System.Web.UI.Page).Controls.Add(CType(myNewLinkButton, System.Web.UI.WebControls.LinkButton))

                       'This is the magical piece.
                       'This solves the problem of the designer not showing up the control.
                       'It will cause the control to show up at designtime on the designer and on the ASPX page
                       'You have to inform the designer of each property that needs to be persisted. The most important is the ID
                       Dim designer As System.Web.UI.Design.ControlDesigner = dcd
                       Dim pd As PropertyDescriptor = TypeDescriptor.GetProperties(myNewLinkButton)("ID")
                       Dim ccea As ComponentChangedEventArgs = New ComponentChangedEventArgs(myNewLinkButton, pd, "", myNewLinkButton.ID)
                       designer.OnComponentChanged(myNewLinkButton, ccea)

                       'Now, the text property
                       pd = TypeDescriptor.GetProperties(myNewLinkButton)("Text")
                       ccea = New ComponentChangedEventArgs(myNewLinkButton, pd, "", myNewLinkButton.Text)
                       designer.OnComponentChanged(myNewLinkButton, ccea)

                       'Finaly Commit
                       myTransaction.Commit()

                       MsgBox("After adding the controls there are " & host.Container.Components.Count.ToString & " controls on the designer")
                       MsgBox("After adding the control there are " & Me.Page.Controls.Count.ToString & " controls on the page")

                       'Select the control on the designer
                       Dim sel As ISelectionService = DirectCast(host.RootComponent.Site.GetService(GetType(ISelectionService)), ISelectionService)
                       sel.SetSelectedComponents(New IComponent() {myNewLinkButton}

From http://www.developmentnow.com/groups/viewthread.aspx?newsgroupid=32&threadid=12360