I do not know if InstallShield has anything available to execute this type
of check.
However, I would recommend that you write a wrapper to accomplish this
task.  Here is some information that should assist in doing this:

Below are the registry locations of SQL Server, and if found, then the
default instance of SQL Server is present:
        For a default instance of the SQL Server 2000 database engine or
MSDE 2000, locate and select this key:
        HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSSQLServer\MSSQLServer\

The CurrentVersion key under that tells the current version of the SQL
Server installed.

        For a named instance of the SQL Server 2000 database engine or
MSDE 2000, locate and select this key:
       HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL
Server\InstanceName\

Under the Microsoft SQL Server there should be at least one instance name.  
Customer can use this information to determine the instance name of the
MSDE installation programatically.

The CurrentVersion key under each instane tells the current version of SQL
Server installed.

========================================

If the customer is curious to know what GUID was used in the MSDE
installation.. read the information in the article below...
http://support.microsoft.com/default.aspx?scid=kb;EN-US;311762

Cheers,
Tim Macaulay, MCSD, MCP.Net
Microsoft Product Support