Lesson 10: Custom Models & C++ SDK

The six built-in flows cover the most common use cases, but DaisyKit's real power is its extensibility. In this lesson we cover:

Converting your own model to NCNN format
Plugging a custom model into ObjectDetectorFlow
Using the C++ graph API to build concurrent multi-model pipelines

1. Converting a Custom Model to NCNN

NCNN supports conversion from most major training frameworks. The workflow is:

Your framework  →  ONNX  →  NCNN (.param + .bin)

PyTorch → NCNN

# Step 1: Export PyTorch model to ONNX
import torch

model = MyModel()
model.load_state_dict(torch.load("model.pth"))
model.eval()

dummy_input = torch.randn(1, 3, 416, 416)
torch.onnx.export(
    model, dummy_input, "model.onnx",
    input_names=["input"],
    output_names=["output"],
    opset_version=11,
)
print("Exported model.onnx")

# Step 2: Convert ONNX to NCNN
pip install onnx
# Install ncnnoptimize from https://github.com/Tencent/ncnn/releases

onnx2ncnn model.onnx model.param model.bin
ncnnoptimize model.param model.bin model-opt.param model-opt.bin 0

TensorFlow/Keras → NCNN

# Step 1: Export to ONNX via tf2onnx
import subprocess
subprocess.run([
    "python", "-m", "tf2onnx.convert",
    "--saved-model", "saved_model/",
    "--output", "model.onnx",
    "--opset", "11",
])

Then use onnx2ncnn as above.

Supported Layer Types

NCNN supports the vast majority of standard layers: Conv2D, BatchNorm, ReLU/SiLU/GELU, Pooling, LSTM, Attention, and more. Check NCNN supported ops if you get unsupported layer errors.

2. Using a Custom YOLOX Model in ObjectDetectorFlow

Once you have my_model.param and my_model.bin:

import json
from daisykit import ObjectDetectorFlow
from daisykit.utils import to_py_type
import cv2

MY_CLASSES = ["cat", "dog", "bird"]  # your custom class names

config = {
    "object_detection_model": {
        "model":   "my_model.param",     # local path, no get_asset_file needed
        "weights": "my_model.bin",
        "input_width": 416,
        "input_height": 416,
        "score_threshold": 0.5,
        "iou_threshold": 0.6,
        "use_gpu": False,
        "class_names": MY_CLASSES,
    },
}

flow = ObjectDetectorFlow(json.dumps(config))

img = cv2.imread("test.jpg")
rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
objects = flow.Process(rgb)
objects_py = to_py_type(objects)

for obj in objects_py:
    name = MY_CLASSES[obj["class_id"]]
    print(f"  {name}: {obj['confidence']:.2f}")

3. The C++ Graph API

For maximum performance, use the C++ SDK directly to build a concurrent graph. Each node runs on its own thread.

Project Setup

git clone --recursive https://github.com/vietanhdev/daisykit
cd daisykit
mkdir build && cd build
cmake .. -DNCNN_VULKAN=ON   # enable GPU support
make -j4

Minimal Face Detection Graph (C++)

#include "daisykitsdk/graphs/core/graph.h"
#include "daisykitsdk/nodes/packet_distributor_node.h"
#include "daisykitsdk/nodes/face_detector_node.h"
#include "daisykitsdk/nodes/facial_landmark_detector_node.h"
#include "daisykitsdk/nodes/face_visualizer_node.h"
#include <opencv2/opencv.hpp>

using namespace daisykit;
using namespace daisykit::graphs;
using namespace daisykit::nodes;

int main() {
    // Create nodes — each runs a dedicated worker thread
    auto distributor = std::make_shared<PacketDistributorNode>(
        "distributor", NodeType::kAsyncNode);

    auto face_detector = std::make_shared<FaceDetectorNode>(
        "face_detector",
        "models/face_detection/yolo_fastest_with_mask/yolo-fastest-opt.param",
        "models/face_detection/yolo_fastest_with_mask/yolo-fastest-opt.bin",
        NodeType::kAsyncNode);

    auto landmark_detector = std::make_shared<FacialLandmarkDetectorNode>(
        "landmark_detector",
        "models/facial_landmark/pfld-sim.param",
        "models/facial_landmark/pfld-sim.bin",
        NodeType::kAsyncNode);

    auto visualizer = std::make_shared<FaceVisualizerNode>(
        "visualizer", NodeType::kAsyncNode, true);

    // Wire up the graph
    //   distributor ─┬─► face_detector ──────────────────► landmark_detector ─► visualizer
    //                └──────────────────────────────────────────────────────────► visualizer
    //                └───────────────────────────────────────────────────────────►(image)
    Graph::Connect(nullptr,          "",        distributor.get(),        "input",  TransmissionProfile(2, true), true);
    Graph::Connect(distributor.get(),"output",  face_detector.get(),      "input",  TransmissionProfile(2, true), true);
    Graph::Connect(distributor.get(),"output",  landmark_detector.get(),  "image",  TransmissionProfile(2, true), true);
    Graph::Connect(face_detector.get(),"output",landmark_detector.get(),  "faces",  TransmissionProfile(2, true), true);
    Graph::Connect(distributor.get(),"output",  visualizer.get(),         "image",  TransmissionProfile(2, true), true);
    Graph::Connect(landmark_detector.get(),"output", visualizer.get(),    "faces",  TransmissionProfile(2, true), true);

    // Activate all nodes (start threads)
    distributor->Activate();
    face_detector->Activate();
    landmark_detector->Activate();
    visualizer->Activate();

    // Feed frames into the graph
    cv::VideoCapture cap(0);
    while (true) {
        cv::Mat frame;
        cap >> frame;
        cv::cvtColor(frame, frame, cv::COLOR_BGR2RGB);

        auto packet = Packet::MakePacket<cv::Mat>(frame);
        distributor->Input("input", packet);

        // Get visualized output
        auto out_packet = visualizer->GetOutput("output");
        if (out_packet) {
            cv::Mat result = out_packet->GetData<cv::Mat>();
            cv::cvtColor(result, result, cv::COLOR_RGB2BGR);
            cv::imshow("Face Detection Graph", result);
        }

        if (cv::waitKey(1) == 'q') break;
    }
    return 0;
}

Why the Graph API Is Faster

In the Python flow, nodes run sequentially: capture → detect → landmark → visualize. In the graph, all four run concurrently:

Frame N:   [capture] ──► [detect]────►[landmark]──►[visualize]
Frame N+1: [capture] ──► [detect]────►[landmark]
Frame N+2: [capture] ──► [detect]
Frame N+3: [capture]

All four nodes process different frames simultaneously on different threads, increasing throughput substantially on multi-core CPUs.

TransmissionProfile

Controls queue behavior between nodes:

TransmissionProfile(
    max_queue_size,  // 2 = drop frames if queue is full (real-time priority)
    drop_on_full     // true = drop oldest packet when queue is full
)

For real-time video, use max_queue_size=2, drop_on_full=true to avoid latency buildup. For batch processing where every frame matters, use a larger queue and drop_on_full=false.

Building Production Systems

For a production deployment combining DaisyKit with a backend service:

# FastAPI server with DaisyKit inference
from fastapi import FastAPI, File, UploadFile
import daisykit, json, cv2, numpy as np
from daisykit.utils import get_asset_file, to_py_type

app = FastAPI()
flow = daisykit.FaceDetectorFlow(json.dumps({...}))

@app.post("/detect-faces")
async def detect_faces(file: UploadFile = File(...)):
    img_bytes = await file.read()
    img_array = np.frombuffer(img_bytes, np.uint8)
    img = cv2.imdecode(img_array, cv2.IMREAD_COLOR)
    rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

    faces = flow.Process(rgb)
    faces_py = to_py_type(faces)

    return {"faces": [
        {"x": f["x"], "y": f["y"], "w": f["w"], "h": f["h"],
         "confidence": f["confidence"], "wearing_mask": f["wearing_mask_prob"] > 0.5}
        for f in faces_py
    ]}

Conclusion

You have now completed the DaisyKit course. Starting from a one-line pip install daisykit, you have:

Run face detection, human pose, background matting, hand pose, and object detection
Built real-time webcam applications in minutes
Deployed those same flows on Android and iOS
Learned how to convert and integrate custom models
Understood the concurrent graph architecture powering DaisyKit's C++ core

Next steps:

Contribute models: Add your trained model to the daisykit-assets registry
Build a flow: Implement a new Flow class in C++ and expose it to Python via pybind11
Explore the GitHub: github.com/vietanhdev/daisykit
Join the community: Open issues, discuss ideas, submit PRs