c++ - .oni ファイルから読み取ると、フレームの順序が正しくありません

Question

私の問題はこれと重複しています。そこに解決策はありませんでした。解決策を見つけて、私の特定の設定を詳しく説明するために、.oni ファイルからフレームを読み取るために使用される関数を以下に示します。この関数が Type == 2 で実行される場合 (つまり、基準が # である RGBD イメージの # に対して実行される)、この関数を for ループで実行すると、ユーザーは各イメージにアクセスできるようになります。ただし、カラーイメージと深度イメージのインデックスは一致せず、順不同です。これは、次の IMG_pipeline::listen(...) へのすべての呼び出しで、waitForAnyStream がタイムアウトするまで続きます。

void IMG_pipeline::listen(int Type, int Criteria){
    int exitNumber;
    clock_t start = clock();
    double elapsedtime;
    openni::VideoFrameRef frame;
    int CurrentIMGCount=0;

    switch (Type){
    case 0:
    {
        exitNumber = -1;
        break;
    }
    case 1:
    {
        exitNumber = Criteria;
        break;
    }
    case 2:
    {
        exitNumber = -1;
        break;
    }
    }


    for (int i = 0;i!=exitNumber;i++){
        readyStream = -1;
        rc = openni::OpenNI::waitForAnyStream(streams, 2, &readyStream, SAMPLE_READ_WAIT_TIMEOUT);
        if (rc != openni::STATUS_OK)
        {
            printf("Wait failed! (timeout is %d ms)\n%s\n", SAMPLE_READ_WAIT_TIMEOUT, openni::OpenNI::getExtendedError());
            //break;
        }

        switch (readyStream)
        {
        case 0:
        {
            // Depth
            depth.readFrame(&frame);
            break;
        }
        case 1:
        {
            // Color
            color.readFrame(&frame);
            break;
        }
        default:
        {
            printf("Unexpected stream: %i\n", readyStream);
            continue;
        }
        }


        int Height = frame.getHeight();
        int Width = frame.getWidth();

        cvColor.release();
        cvX.release();
        cvY.release();
        cvZ.release();

        cvColor = cv::Mat(Height, Width, CV_8UC3);
        cvX = cv::Mat(Height, Width, CV_32F);
        cvY = cv::Mat(Height, Width, CV_32F);
        cvZ = cv::Mat(Height, Width, CV_32F);

        switch (frame.getVideoMode().getPixelFormat())
        {
        case openni::PIXEL_FORMAT_DEPTH_1_MM:
        case openni::PIXEL_FORMAT_DEPTH_100_UM:
        {
            openni::DepthPixel* pDepth = (openni::DepthPixel*)frame.getData();
            int k =0;
            for (int ri = 0; ri<Height; ri++)
            {
                for (int ci = 0; ci<Width; ci++)
                {
                    float pdepth_val = pDepth[k];
                    openni::CoordinateConverter::convertDepthToWorld(depth, (float)ri, (float)ci, pdepth_val, &cvX.at<float>(ri,ci), &cvY.at<float>(ri,ci), &cvZ.at<float>(ri,ci));
                    k++;
                }
            }
            TotalFrames[0]++;
            XYZCaptured = true;
            printf("Frame Index: %i \n", frame.getFrameIndex());
            printf("Depth Captured. \n");
            break;
        }
        case openni::PIXEL_FORMAT_RGB888:
        {
            cvColor.data = (uchar*)frame.getData();
            TotalFrames[1]++;
            ColorCaptured = true;
            printf("Frame Index: %i \n", frame.getFrameIndex());
            printf("Color Captured. \n");
            break;
        }
        default:
            printf("Unknown format \n");
        }

        printf("Frame extracted. \n");
        if (ColorCaptured && XYZCaptured){
            if (NewButNotRead == true){
                IMGsMissed++;


            }
            else
                NewButNotRead = true;

            ColorCaptured = false;
            XYZCaptured = false;
            RGBD_out.clear();
            RGBD_out.push_back(cvX);
            RGBD_out.push_back(cvY);
            RGBD_out.push_back(cvZ);
            RGBD_out.push_back(cvColor);
            CurrentIMGCount++;
            printf("Image overwritten. \n");

        }
        elapsedtime=(clock()-start)/((double)CLOCKS_PER_SEC);

        printf("Time since listen initiation: %f \n \n", elapsedtime);
        if (CurrentIMGCount ==Criteria && Type == 2)
            return;
        else if (elapsedtime>(double)Criteria && Type==0)
            return;

    }
    frame.release();

}

コンソール出力の例を次に示します。

Frame Index: 1 
Depth Captured. 
Frame extracted. 
Time since listen initiation: 0.004846 

Frame Index: 2 
Depth Captured. 
Frame extracted. 
Time since listen initiation: 0.011601 

Frame Index: 1 
Color Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.012640 

Depth frame count: 3 
Color frame count: 2 
Frame Index: 54 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000067 

Frame Index: 57 
Depth Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.005878 

Depth frame count: 4 
Color frame count: 3 
Frame Index: 96 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000079 

Frame Index: 99 
Depth Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.003628 

Depth frame count: 5 
Color frame count: 4 
Frame Index: 126 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000048 

Frame Index: 130 
Depth Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.004782 

Depth frame count: 6 
Color frame count: 5 
Frame Index: 152 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000065 

Frame Index: 156 
Depth Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.008294 

Depth frame count: 7 
Color frame count: 6 
Frame Index: 181 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000045 

Frame Index: 185 
Depth Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.004095 

Depth frame count: 8 
Color frame count: 7 
Frame Index: 208 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000054 

Frame Index: 212 
Depth Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.004242 

Depth frame count: 9 
Color frame count: 8 
Frame Index: 236 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000092 

Frame Index: 240 
Depth Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.005918 

Depth frame count: 10 
Color frame count: 9 
Frame Index: 261 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000731 

Frame Index: 262 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000877 

Frame Index: 266 
Depth Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.009347 

Depth frame count: 11 
Color frame count: 11 
Frame Index: 286 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000047 

Frame Index: 290 
Depth Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.006080 

Depth frame count: 12 
Color frame count: 12 
Frame Index: 311 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000072 

Frame Index: 315 
Depth Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.006453 

Depth frame count: 13 
Color frame count: 13 
Frame Index: 337 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000062 

Frame Index: 341 
Depth Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.007485 

Depth frame count: 14 
Color frame count: 14 
Frame Index: 367 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000042 

Frame Index: 371 
Depth Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.003758 

Depth frame count: 15 
Color frame count: 15 
Frame Index: 390 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000073 

Frame Index: 395 
Depth Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.007917 

Depth frame count: 16 
Color frame count: 16 
Frame Index: 416 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000105 

Frame Index: 421 
Depth Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.007554 

Depth frame count: 17 
Color frame count: 17 
Frame Index: 453 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000060 

Frame Index: 458 
Depth Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.006150 

Depth frame count: 18 
Color frame count: 18 
Frame Index: 481 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000074 

Frame Index: 486 
Depth Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.007169 

Depth frame count: 19 
Color frame count: 19 
Frame Index: 517 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000045 

Frame Index: 522 
Depth Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.004196 

Depth frame count: 20 
Color frame count: 20 
Frame Index: 547 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000071 

Frame Index: 552 
Depth Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.007375 

Depth frame count: 21 
Color frame count: 21 
Frame Index: 625 
Color Captured. 
Frame extracted. 
Time since listen initiation: 0.000179 

Frame Index: 631 
Depth Captured. 
Frame extracted. 
Image overwritten. 
Time since listen initiation: 0.007922 

Depth frame count: 22 
Color frame count: 22 
Wait failed! (timeout is 2000 ms)
    waitForStreams: timeout reached

Unexpected stream: -1
Wait failed! (timeout is 2000 ms)
    waitForStreams: timeout reached

Unexpected stream: -1
Wait failed! (timeout is 2000 ms)
    waitForStreams: timeout reached

Unexpected stream: -1
Wait failed! (timeout is 2000 ms)
    waitForStreams: timeout reached

Unexpected stream: -1
Wait failed! (timeout is 2000 ms)
    waitForStreams: timeout reached

Unexpected stream: -1
Wait failed! (timeout is 2000 ms)
    waitForStreams: timeout reached

Unexpected stream: -1
Wait failed! (timeout is 2000 ms)
    waitForStreams: timeout reached

Unexpected stream: -1

IMG_pipeline::listen(...) の呼び出しは次のとおりです。

IMG_pipeline pip_inst;
std::string FileName = "/home/derek/Test Data/RGBD/RGBD_S2_R1";
int Type = 2;
int Criteria = 1;
std::vector<cv::Mat> OUT;
int NumMissedIMGs;

int Start;
int Stop;

pip_inst.connect(FileName);

while (true)
{
  pip_inst.listen(Type, Criteria);
  if (pip_inst.IsNewIMG()){
    OUT = pip_inst.GetImage();
    cv::imshow("Current Frame", OUT.at(3));
    char c = cv::waitKey(0);
    if (c == 'f')
    {
      printf("Depth frame count: %i \n", pip_inst.GetDepthFrameCount());
      printf("Color frame count: %i \n", pip_inst.GetColorFrameCount());

    }
    else
    {
      Start = pip_inst.GetColorFrameCount();
      break;
    }
    cv::destroyWindow("Current Frame");

  }


}

カラー画像も、R、G、B の色合いが交互になっています。ただし、データが cv::Mat にある順序に問題があると確信しています。

さらに興味深いことに、多くのフレームを通過する IMG_pipeline::listen(...) の呼び出しでは、IMG_pipeline::listen(...) を複数回実行して .oni ファイルをインクリメントすると、異なるインデックス結果が得られます。

score 0 · Accepted Answer

そのため、コンパイラごとに uchar が異なるため、RGB 情報が正しくコピーされませんでした。そのため、RGB 情報の case ステートメントは次のように変更されました。

case openni::PIXEL_FORMAT_RGB888:
        {
            openni::RGB888Pixel* imgbuffer = (openni::RGB888Pixel*)frame.getData();
            //cvColor.data = (uchar*)imgbuffer;
            memcpy( cvColor.data, imgbuffer, 3*frame.getHeight()*frame.getWidth()*sizeof(uint8_t));
            cv::cvtColor(cvColor,cvColor,cv::COLOR_BGR2RGB);
            TotalFrames[1]++;
            ColorCaptured = true;
            printf("Frame Index: %i \n", frame.getFrameIndex());
            printf("Color Captured. \n");
            break;
        }

さらに、すべてのフレームをキャプチャできるようにするために、ファイルの再生速度を遅くしました。興味深いことに、.oni ファイルを開くと、フレームインデックスを取得するのではなく、一定の速度で再生されます (これは、キャプチャされた速度であるとしか思えません)。とにかく、これはで行われます

Source.getPlaybackControl()->setSpeed(Ratio);

Source は自分のデバイスで、Ratio はユーザー指定の float です。これが将来誰かに役立つことを願っています。

c++ - .oni ファイルから読み取ると、フレームの順序が正しくありません

2 に答える 2

Related

Reference