Programming Web Camera for Windows

Web cameras, like most other PC components, have many brands and models, so generally speaking, it is essential to program a Web camera by using API's supported widely by Web camera drivers to make the application accommodate a wide range of cameras.. Programming a Web camera through a proprietary driver will not be discussed here.

Fortunately, Microsoft offers models or high-level drivers supported by low-level drivers of manufacturers to integrate the cameras with Windows.

1. Windows Image Acquisition (WIA) API is designed for use by C/C++ programmers. WIA Automation Layer componentizes the WIA API so it can be easily used by Visual Basic, ASP, C#, and scripting languages. If one chooses the WIA route for programming a Web camera, this automation layer should be the best way for most purposes.

2. Windows Portable Device is the latest way of programming a Web camera. This system supersedes both Windows Media Device Manager and Windows Image Acquisition by providing a flexible, robust way for a computer to communicate with music players, storage devices, mobile phones, cameras, and many other types of connected devices. It is supported by Windows Vista by default. Windows XP needs to install the Windows Media Format runtime to support Windows Portable Device. Although WPD API is a set of COM interfaces, one can supposedly use it relatively easily with an IDE and languages supporting COM, such as C#, VB in Visual Studio, but unfortunately, a lot of cumbersome coding needs to be done to use it in languages such as C#. If one decides to take this route, a good starting point is this blog.

3, Windows Media Encoder is the best for encoding and broadcasting media files and streams, but it is very limited in terms of media rendering.

3. DirectShow, which used to be a part of DirectX but is now a part of Windows SDK, is the most powerful way for programming a Webcam. It can do almost everything the above three are capable of doing. It was made for C++. There are quite a few efforts to wrap DirectShow so it can be used by managed code such as C#, but the stability and completeness of those wrappers are questionable. Now, DirectX supports managed code, and this may be the reason that DirectShow has been kicked out of DirectX. The reason MS does not make wrappers for DirectShow may be the concern for performance.

The following discussion is for using DirectShow.

Create a DirectShow filter graph to render a webcam capture

Creating a windowed mode display of webcam video is relatively easy, but its user interface is poor. Windowless mode is the way for any serious application. To do this, the following steps need to be taken:

1. Create graph builder (CLSID_FilterGraph).
2. Create capture graph builder(CLSID_CaptureGraphBuilder2).
3. Set the capture graph builder to use the created graph builder.
4. Obtain the media control for controlling the video play(IID_IMediaControl).
5. Set up VMR for windowless rendering with the following steps
   1. Create VMR (e.g., CLSID_VideoMixingRenderer)
   2. Add the VMR to the graph builder
   3. Get the VMR configuration interface(IVMRFilterConfig) to configure the mode to VMRMode_Windowless)
   4. Get the VMR windowless control (IID_IVMRWindowlessControl) to set the rendering window
6. Create the webcam filter graph and add it to the graph builder.
7. Use the capture graph builder to build the graph (i.e., link the Webcam to the VMR).
8. Use the media control to play the video

Stream data remotely

To stream video data remotely is a long story due to the incoherent multiple solutions provided by Microsoft. One can set up video data streaming by using Windows Media Encoder in minutes, but the broadcast is a public broadcast that everyone can enjoy. One can use Windows Media Player SDK and Windows Media Services SDK to do the job, but they pose limitations on both client side and server side, impose installation of additional components, and require quite some learning, though only a tiny fraction of their functions are needed. The Multimedia Streaming (a set of API's that enable an application to get audio and video data from a DirectShow filter graph without writing a custom DirectShow filter) is a bit deceiving. It is either impossible or very difficult to get the raw data.

The best way to stream video data remotely with full control is to write a DirectShow filter to connect to the filters from which data will be streamed.

There are two ways to write a DirectShow filter for this purpose:

1. Write a filter that is a true COM object, register it, and use it just like other normal DirectShow filter. When the application is distributed, the filter also needs to be distributed.

2. Write a filter as one of the classes of the application. In this case, just like the first method. In this case, one does not need to worry about registration or distribution of the COM object.

The Windows SDK document covers the first method very well. The second method will be briefly covered here.

Let us call the second type an in-application filter. The filter class also needs to be derived from DirectShow Base Classes that come as sample code with the Windows SDK. Suppose the application project is a C++ project. The configuration of the project should follow the instructions of the Building DirectShow Filters chapter of the SDK document, except for the parts related to the DLL, and ignore the default libraries. The DLL part should be ignored because the application is not a DLL. The default library should not be ignored. If msvcrt.lib gives trouble, it should be ignored by using the "ignore Specific Library" option. The library file winmm.lib should be added as an additional dependency.

The sending (server) side:

The following are specific steps to create the in-application filter to grab sample data:

1. Derive a class from CVideoTransformFilter and implement the five virtual functions of CVideoTransformFilter (CheckInputType(), CheckTransform(), DecideBufferSize(), GetMediaType(), Transform()). Suppose the new class name is CVideoDataProvider for the sake this discussion.

2. Suppose the class that creates the filter graph is called CMyView. Pass a pointer of the instance of CMyView to the CVideoDataProvider instance. This can be done through the constructor. This pointer will be used to access objects of CMyView to handle data. A callback function can be used for data transferring, but I always try to avoid using callback functions because there is no elegant way to safely use callback functions in C++. Callback functions can be used elegantly for C applications.

3. Override the Transform() of CVideoTransformFilter to copy the data from the input pin to the output pin and use the pointer of CMyView to access any objects of CMyView to transfer data. One way to do this is to add the new data to a queue. Since most C++ classes are not thread-safe, synchronization objects such as CCriticalSection should be used to ensure simultaneous access to an object by multiple threads. Only the sample data needs to be copied every time new data arrives. Other data, such as input pin properties (media type, prerol property, etc.), need to be copied only once. Since this filter does not do anything to the data before transferring it to the output pin, one would think CTransInPlaceFilter is a more appropriate class to derive this class from because it will save copying data from one buffer to another. It has been shown that data copying takes very little resources for any contemporary PC. Using CVideoTransformFilter offers the opportunity to verify that the transferred data is valid and adequate. One can create a source filter and feed exactly the same data as copied to the output pin of CVideoDataProvider. If CVideoDataProvider works, the source filter should work. If not, the cause should lie with the newly created filters.

Without compression, the sample size of webcam capture can be very large. For 640 x 480 resolution, the sample size could be about 1 MB. The sample can be 30 frames/second. Apparently, transferring data at such a rate over the network, especially the Internet, is impractical in most places. Compression is necessary. One can use "Microsoft Windows Media Video 9".

The receiving (client) side: