Taking pictures with Kinect & calling GDI+ to reduce the file size
Hi Community,
As mentioned in my previous post, I’ve been putting together an application which comprises technologies like Kinect, Windows Azure & Visual C++. The application name is “CyberNanny” and I wrote it for my beautiful newborn baby 
. Actually, I’ve submitted today an article to MSDN magazine about the aforementioned solution.
Today’s post is about how we can take pictures with the Kinect sensor and reduce the size of the resulting image file (for emailing purposes).
Let’s assume that we have a smart pointer of type INuiSensor. INuiSensor is the COM representation of the physical sensor. We also have another smart pointer that holds a reference to a custom class called DrawDevice that performs the drawing via Direct2D and ID2D1Factory
CComPtr<INuiSensor> m_pSensor;
std::shared_ptr<DrawDevice> m_pDrawColor;
void Nui_Core::TakePicture(std::shared_ptr<BYTE>& imageBytes, int& bytesCount) {
byte *bytes;
NUI_IMAGE_FRAME imageFrame;
NUI_LOCKED_RECT LockedRect;
if (SUCCEEDED(m_pSensor->NuiImageStreamGetNextFrame(m_hVideoStream, m_millisecondsToWait, &imageFrame))) {
auto pTexture = imageFrame.pFrameTexture;
pTexture->LockRect(0, &LockedRect, NULL, 0);
if (LockedRect.Pitch != 0) {
bytes = static_cast<BYTE *>(LockedRect.pBits);
m_pDrawColor->Draw(bytes, LockedRect.size);
}
pTexture->UnlockRect(0);
imageBytes.reset(new BYTE[LockedRect.size]);
memcpy(imageBytes.get(), bytes, LockedRect.size);
bytesCount = LockedRect.size;
m_pSensor->NuiImageStreamReleaseFrame(m_hVideoStream, &imageFrame );
}
}
Please note that we deal with a pointer to byte (array of bytes) and it’s not a safe practice to return a pointer because the memory address might change and our pointer might point to something else, therefore we pass a smart pointer by reference to store the bytes in it.
The code to handcraft our image (picture) which is a BMP is shown below
std::wstring ImageFile::SerializeImage(std::shared_ptr<BYTE>& bytes, int byteCount) {
RPC_WSTR guidAsStr;
WCHAR tempPath[MAX_PATH];
std::wstring retval, tempFile;
std::unique_ptr<GUID> guid(new GUID);
CoCreateGuid(guid.get());
UuidToString(guid.get(), &guidAsStr);
auto str = std::wstring((LPTSTR) guidAsStr);
GetTempPath(MAX_PATH, tempPath);
tempFile = std::wstring(tempPath).append(L"photo_").append(str).append(L".bmp");
auto infoHeader = GetBitmapInfoHeader();
auto fileHeader = GetBitmapFileHeader(infoHeader);
if (SerializeImageHelper(infoHeader, fileHeader, tempFile, bytes, byteCount)) {
if (RotateImage(tempFile, retval))
m_savedImagePath = retval;
} else retval = L"";
return retval;
}
BITMAPINFOHEADER ImageFile::GetBitmapInfoHeader() {
BITMAPINFOHEADER retval = {0};
retval.biSize = sizeof(BITMAPINFOHEADER);
retval.biBitCount = 32;
retval.biPlanes = 1;
retval.biCompression = BI_RGB;
retval.biWidth = 640;
retval.biHeight = 480;
retval.biSizeImage = ((((retval.biWidth * retval.biBitCount) + 31) & ~31) >> 3) * retval.biHeight;
return retval;
}
BITMAPFILEHEADER ImageFile::GetBitmapFileHeader(const BITMAPINFOHEADER& infoHeader) {
BITMAPFILEHEADER retval = {0};
auto nBitsOffset = sizeof(BITMAPFILEHEADER) + infoHeader.biSize;
auto lImageSize = infoHeader.biSizeImage;
auto lFileSize = nBitsOffset + lImageSize;
retval.bfType = 'B'+('M'<<8);
retval.bfOffBits = nBitsOffset;
retval.bfSize = lFileSize;
return retval;
}
bool ImageFile::SerializeImageHelper(const BITMAPINFOHEADER& infoHeader, const BITMAPFILEHEADER& fileHeader,
std::wstring& filePath, std::shared_ptr<BYTE>& bytes, int byteCount) {
auto retval = false;
auto hFile = CreateFile(filePath.c_str(), GENERIC_WRITE, FILE_SHARE_READ, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
if (hFile != INVALID_HANDLE_VALUE) {
auto newFile = CFile(hFile);
newFile.Write(&fileHeader, sizeof(BITMAPFILEHEADER));
newFile.Write(&infoHeader, sizeof(BITMAPINFOHEADER));
newFile.Write(bytes.get(), byteCount);
newFile.Close();
retval = true;
}
return retval;
}
In order to save our BMP, we must specify two things before serializing the bytes:
Another detail to take into account is, the resulting bitmap is approximately 1.7 MB in size and it’s upside down – This is not very convenient for emailing purposes, so we have to rotate the image 180 degrees and save it with a different format (Encode it), the snippet below shows how we can do this
bool ImageFile::RotateImage(const std::wstring& filePath, std::wstring& newFile) {
CLSID jpgClsid;
newFile = filePath;
auto retval = false;
ULONG_PTR gdiplusToken;
GdiplusStartupInput gdiplusStartupInput;
GdiplusStartup(&gdiplusToken, &gdiplusStartupInput, NULL);
if ((retval = GetEncoderClsid(L"image/jpeg", &jpgClsid))) {
auto fileToRotate = new Image(filePath.c_str(), FALSE);
newFile.replace(newFile.find(L"bmp"), 3, L"jpg");
fileToRotate->RotateFlip(Rotate180FlipNone);
fileToRotate->Save(newFile.c_str(), &jpgClsid, NULL);
delete fileToRotate;
}
::DeleteFile(filePath.c_str());
GdiplusShutdown(gdiplusToken);
return retval;
}
bool ImageFile::GetEncoderClsid(const WCHAR* format, CLSID* pClsid) {
UINT num = 0;
UINT size = 0;
auto retval = false;
ImageCodecInfo* pImageCodecInfo = NULL;
if ((GetImageEncodersSize(&num, &size)) == Status::Ok && (pImageCodecInfo = (ImageCodecInfo*)(malloc(size))) != NULL) {
if (GetImageEncoders(num, size, pImageCodecInfo) == Status::Ok) {
for(auto index = 0; index < num; ++index) {
if(wcscmp(pImageCodecInfo[index].MimeType, format) == 0 ) {
*pClsid = pImageCodecInfo[index].Clsid;
free(pImageCodecInfo);
pImageCodecInfo = NULL;
retval = true;
break;
}
}
}
}
if (!pImageCodecInfo)
free(pImageCodecInfo);
return retval;
}
As you can see, we reduce the file size by saving the image as a JPEG by calling the Save method of the Image class (from 1.7 MB the file size was reduced to 32 KB) and we rotate the image by calling the RotateFlip method of the same class. This functionality is provided by GDI+. It’s important to note that GDI+ needs to be initialized and then shutdown once we’ve finished using it.
Best Regards,
Angel
Filed under: Microsoft - @ April 17, 2012 4:12 pm
Tags: C++, COM, GDI+, Kinect, Visual C++