Here is the log:
jetson-containers run $(autotag nano_llm) python3 -m nano_llm.agents.video_query --api=mlc --model Efficient-Large-Model/VILA1.5-3b --max-context-len 256 --max-new-tokens 32 --video-input /dev/video1 --video-output webrtc://@:8554/output
Namespace(packages=['nano_llm'], prefer=['local', 'registry', 'build'], disable=[''], user='dustynv', output='/tmp/autotag', quiet=False, verbose=False) -- L4T_VERSION=36.4.3 JETPACK_VERSION=6.2 CUDA_VERSION=12.6 -- Finding compatible container image for ['nano_llm'] dustynv/nano_llm:r36.4.0 V4L2_DEVICES: --device /dev/video1 --device /dev/video2 ### DISPLAY environmental variable is already set: ":1" localuser:root being added to access control list ### ARM64 architecture detected ### Jetson Detected SYSTEM_ARCH=tegra-aarch64 + docker run --runtime nvidia --env NVIDIA_DRIVER_CAPABILITIES=compute,utility,graphics -it --rm --network host --shm-size=8g --volume /tmp/argus_socket:/tmp/argus_socket --volume /etc/enctune.conf:/etc/enctune.conf --volume /etc/nv_tegra_release:/etc/nv_tegra_release --volume /tmp/nv_jetson_model:/tmp/nv_jetson_model --volume /var/run/dbus:/var/run/dbus --volume /var/run/avahi-daemon/socket:/var/run/avahi-daemon/socket --volume /var/run/docker.sock:/var/run/docker.sock --volume /ssd/jetson-containers/data:/data -v /etc/localtime:/etc/localtime:ro -v /etc/timezone:/etc/timezone:ro --device /dev/snd -e PULSE_SERVER=unix:/run/user/1000/pulse/native -v /run/user/1000/pulse:/run/user/1000/pulse --device /dev/bus/usb -e DISPLAY=:1 -v /tmp/.X11-unix/:/tmp/.X11-unix -v /tmp/.docker.xauth:/tmp/.docker.xauth -e XAUTHORITY=/tmp/.docker.xauth --device /dev/video1 --device /dev/video2 --device /dev/i2c-0 --device /dev/i2c-1 --device /dev/i2c-2 --device /dev/i2c-4 --device /dev/i2c-5 --device /dev/i2c-7 --device /dev/i2c-9 --name jetson_container_20250618_213716 dustynv/nano_llm:r36.4.0 python3 -m nano_llm.agents.video_query --api=mlc --model Efficient-Large-Model/VILA1.5-3b --max-context-len 256 --max-new-tokens 32 --video-input /dev/video1 --video-output webrtc://@:8554/output /usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py:124: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead. warnings.warn( /usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1142: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. warnings.warn( Fetching 13 files: 100%|██████████████████████████████████████████████████████████████████████████████████████| 13/13 [00:00<00:00, 61059.30it/s] Fetching 17 files: 100%|██████████████████████████████████████████████████████████████████████████████████████| 17/17 [00:00<00:00, 26023.05it/s] 21:37:33 | INFO | loading /data/models/huggingface/models--Efficient-Large-Model--VILA1.5-3b/snapshots/42d1dda6807cc521ef27674ca2ae157539d17026 with MLC 21:37:38 | INFO | NumExpr defaulting to 6 threads. 21:37:39 | WARNING | AWQ not installed (requires JetPack 6 / L4T R36) - AWQ models will fail to initialize 21:37:40 | INFO | patching model config with {'model_type': 'llama'} 21:37:42 | INFO | device=cuda(0), name=Orin, compute=8.7, max_clocks=1020000, multiprocessors=8, max_thread_dims=[1024, 1024, 64], api_version=12060, driver_version=None 21:37:42 | INFO | loading VILA1.5-3b from /data/models/mlc/dist/VILA1.5-3b/ctx256/VILA1.5-3b-q4f16_ft/VILA1.5-3b-q4f16_ft-cuda.so 21:37:42 | WARNING | model library /data/models/mlc/dist/VILA1.5-3b/ctx256/VILA1.5-3b-q4f16_ft/VILA1.5-3b-q4f16_ft-cuda.so was missing metadata 21:37:43 | INFO | loading siglip vision model /data/models/huggingface/models--Efficient-Large-Model--VILA1.5-3b/snapshots/42d1dda6807cc521ef27674ca2ae157539d17026/vision_tower 21:37:48 | WARNING | disabling CLIP with TensorRT due to limited memory (falling back to Transformers API) 21:37:48 | INFO | loaded siglip vision model /data/models/huggingface/models--Efficient-Large-Model--VILA1.5-3b/snapshots/42d1dda6807cc521ef27674ca2ae157539d17026/vision_tower 21:37:48 | INFO | mm_projector (mlp_downsample) Sequential( (0): DownSampleBlock() (1): LayerNorm((4608,), eps=1e-05, elementwise_affine=True) (2): Linear(in_features=4608, out_features=2560, bias=True) (3): GELU(approximate='none') (4): Linear(in_features=2560, out_features=2560, bias=True) ) 21:37:48 | INFO | mm_projector weights: dict_keys(['1.bias', '1.weight', '2.bias', '2.weight', '4.bias', '4.weight']) ┌────────────────────────────┬─────────────────────────────────────────────────────────────────────────────┐ │ _name_or_path │ ./llm │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ architectures │ ['LlamaForCausalLM'] │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ drop_path_rate │ 0.0 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ hidden_size │ 2560 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ image_aspect_ratio │ resize │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ interpolate_mode │ linear │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ mm_hidden_size │ 1152 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ mm_projector_lr │ │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ mm_use_im_patch_token │ False │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ mm_use_im_start_end │ False │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ mm_vision_select_feature │ cls_patch │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ mm_vision_select_layer │ -2 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ model_dtype │ torch.bfloat16 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ model_type │ llama │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ num_video_frames │ 8 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ resume_path │ ./vlm │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ s2 │ False │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ s2_max_split_size │ 336 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ s2_scales │ 336,672,1008 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ transformers_version │ 4.36.2 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ tune_language_model │ True │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ tune_mm_projector │ True │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ tune_vision_tower │ True │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ vision_resolution │ -1 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ name │ VILA1.5-3b │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ api │ mlc │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ max_position_embeddings │ 4096 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ mm_vision_tower │ /data/models/huggingface/models--Efficient-Large-Model--VILA1.5-3b/snapshot │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ mm_projector_path │ /data/models/huggingface/models--Efficient-Large-Model--VILA1.5-3b/snapshot │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ mm_projector_type │ mlp_downsample │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ attention_bias │ False │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ attention_dropout │ 0.0 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ bos_token_id │ 1 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ eos_token_id │ 2 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ hidden_act │ silu │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ initializer_range │ 0.02 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ intermediate_size │ 6912 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ model_max_length │ 4096 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ num_attention_heads │ 20 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ num_hidden_layers │ 32 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ num_key_value_heads │ 20 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ pad_token_id │ 0 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ pretraining_tp │ 1 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ rms_norm_eps │ 1e-05 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ rope_scaling │ │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ rope_theta │ 10000.0 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ tie_word_embeddings │ False │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ tokenizer_model_max_length │ 4096 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ tokenizer_padding_side │ right │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ torch_dtype │ bfloat16 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ use_cache │ True │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ vocab_size │ 32000 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ quant │ q4f16_ft │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ type │ llama │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ max_length │ 256 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ prefill_chunk_size │ -1 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ load_time │ 14.92252376800002 │ ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ params_size │ 1300.8330078125 │ └────────────────────────────┴─────────────────────────────────────────────────────────────────────────────┘ 21:37:48 | INFO | using chat template 'vicuna-v1' for model VILA1.5-3b 21:37:48 | INFO | model 'VILA1.5-3b', chat template 'vicuna-v1' stop tokens: ['</s>'] -> [2] 21:37:48 | INFO | Warming up LLM with query 'What is 2+2?' 21:37:49 | INFO | Warmup response: '4</s>' 21:37:49 | INFO | plugin | connected PrintStream to on_text on channel 0 21:37:49 | INFO | plugin | connected ChatQuery to PrintStream on channel 0 huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks... To disable this warning, you can either: - Avoid using `tokenizers` before the fork if possible - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false) (gst-plugin-scanner:73): GLib-GObject-WARNING **: 21:37:51.179: cannot register existing type 'GstRtpSrc' (gst-plugin-scanner:73): GLib-GObject-CRITICAL **: 21:37:51.179: g_type_add_interface_static: assertion 'G_TYPE_IS_INSTANTIATABLE (instance_type)' failed (gst-plugin-scanner:73): GLib-CRITICAL **: 21:37:51.179: g_once_init_leave: assertion 'result != 0' failed (gst-plugin-scanner:73): GStreamer-CRITICAL **: 21:37:51.180: gst_element_register: assertion 'g_type_is_a (type, GST_TYPE_ELEMENT)' failed (gst-plugin-scanner:73): GLib-GObject-WARNING **: 21:37:51.180: cannot register existing type 'GstRtpSink' (gst-plugin-scanner:73): GLib-GObject-CRITICAL **: 21:37:51.180: g_type_add_interface_static: assertion 'G_TYPE_IS_INSTANTIATABLE (instance_type)' failed (gst-plugin-scanner:73): GLib-CRITICAL **: 21:37:51.180: g_once_init_leave: assertion 'result != 0' failed (gst-plugin-scanner:73): GStreamer-CRITICAL **: 21:37:51.180: gst_element_register: assertion 'g_type_is_a (type, GST_TYPE_ELEMENT)' failed sh: 1: lsmod: not found sh: 1: modprobe: not found (Argus) Error EndOfFile: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 277) (Argus) Error EndOfFile: Receive worker failure, notifying 1 waiting threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 350) (Argus) Error InvalidState: Argus client is exiting with 1 outstanding client threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 366) (Argus) Error EndOfFile: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 379) (Argus) Error EndOfFile: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 145) (Argus) Error EndOfFile: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 92) [gstreamer] initialized gstreamer, version 1.20.3.0 [gstreamer] gstCamera -- attempting to create device v4l2:///dev/video1 [gstreamer] gstCamera -- found v4l2 device: Logitech StreamCam [gstreamer] v4l2-proplist, device.path=(string)/dev/video1, udev-probed=(boolean)false, device.api=(string)v4l2, v4l2.device.driver=(string)uvcvideo, v4l2.device.card=(string)"Logitech\ StreamCam", v4l2.device.bus_info=(string)usb-3610000.usb-2.4, v4l2.device.version=(uint)331668, v4l2.device.capabilities=(uint)2225078273, v4l2.device.device_caps=(uint)69206017; [gstreamer] gstCamera -- found 20 caps for v4l2 device /dev/video1 [gstreamer] [0] video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)5/1; [gstreamer] [1] video/x-raw, format=(string)YUY2, width=(int)1600, height=(int)896, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 15/2, 5/1 }; [gstreamer] [2] video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 10/1, 15/2, 5/1 }; [gstreamer] [3] video/x-raw, format=(string)YUY2, width=(int)960, height=(int)540, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [4] video/x-raw, format=(string)YUY2, width=(int)848, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [5] video/x-raw, format=(string)YUY2, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [6] video/x-raw, format=(string)YUY2, width=(int)640, height=(int)360, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [7] video/x-raw, format=(string)YUY2, width=(int)424, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [8] video/x-raw, format=(string)YUY2, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [9] video/x-raw, format=(string)YUY2, width=(int)176, height=(int)144, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [10] image/jpeg, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [11] image/jpeg, width=(int)1600, height=(int)896, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [12] image/jpeg, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [13] image/jpeg, width=(int)960, height=(int)540, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [14] image/jpeg, width=(int)848, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [15] image/jpeg, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [16] image/jpeg, width=(int)640, height=(int)360, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [17] image/jpeg, width=(int)424, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [18] image/jpeg, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [19] image/jpeg, width=(int)176, height=(int)144, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] gstCamera -- selected device profile: codec=MJPEG format=unknown width=1280 height=720 framerate=30 [gstreamer] gstCamera pipeline string: [gstreamer] v4l2src device=/dev/video1 do-timestamp=true ! image/jpeg, width=(int)1280, height=(int)720, framerate=30/1 ! jpegdec name=decoder ! video/x-raw ! appsink name=mysink sync=false [gstreamer] gstCamera successfully created device v4l2:///dev/video1 [video] created gstCamera from v4l2:///dev/video1 ------------------------------------------------ gstCamera video options: ------------------------------------------------ -- URI: v4l2:///dev/video1 - protocol: v4l2 - location: /dev/video1 - port: 1 -- deviceType: v4l2 -- ioType: input -- codec: MJPEG -- codecType: cpu -- width: 1280 -- height: 720 -- frameRate: 30 -- numBuffers: 4 -- zeroCopy: true -- flipMethod: none -- sslCert /etc/ssl/private/localhost.cert.pem -- sslKey /etc/ssl/private/localhost.key.pem ------------------------------------------------ [gstreamer] gstEncoder -- codec not specified, defaulting to H.264 failed to find/open file /proc/device-tree/model [gstreamer] gstEncoder -- detected board 'NVIDIA Jetson Orin Nano Engineering Reference Developer Kit Super' [gstreamer] gstEncoder -- hardware encoder not detected, reverting to CPU encoder [gstreamer] gstEncoder -- pipeline launch string: [gstreamer] appsrc name=mysource is-live=true do-timestamp=true format=3 ! x264enc name=encoder bitrate=4000 speed-preset=ultrafast tune=zerolatency key-int-max=30 insert-vui=1 ! video/x-h264 ! rtph264pay config-interval=1 ! application/x-rtp,media=video,encoding-name=H264,clock-rate=90000,payload=96 ! tee name=videotee ! queue ! fakesink [webrtc] WebRTC server started @ https://nano:8554 [webrtc] WebRTC server thread running... [webrtc] websocket route added /output [video] created gstEncoder from webrtc://@:8554/output ------------------------------------------------ gstEncoder video options: ------------------------------------------------ -- URI: webrtc://@:8554/output - protocol: webrtc - location: 0.0.0.0 - port: 8554 -- deviceType: ip -- ioType: output -- codec: H264 -- codecType: cpu -- frameRate: 30 -- bitRate: 4000000 -- numBuffers: 4 -- zeroCopy: true -- latency 10 -- sslCert /etc/ssl/private/localhost.cert.pem -- sslKey /etc/ssl/private/localhost.key.pem ------------------------------------------------ 21:37:53 | INFO | plugin | connected VideoSource to on_video on channel 0 21:37:53 | INFO | mounting webserver path /data/datasets/uploads to /images/uploads [gstreamer] opening gstCamera for streaming, transitioning pipeline to GST_STATE_PLAYING 21:37:53 | INFO | starting webserver @ https://0.0.0.0:8050 [gstreamer] gstreamer changed state from NULL to READY ==> mysink [gstreamer] gstreamer changed state from NULL to READY ==> capsfilter1 [gstreamer] gstreamer changed state from NULL to READY ==> decoder [gstreamer] gstreamer changed state from NULL to READY ==> capsfilter0 [gstreamer] gstreamer changed state from NULL to READY ==> v4l2src0 [gstreamer] gstreamer changed state from NULL to READY ==> pipeline0 [gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter1 [gstreamer] gstreamer changed state from READY to PAUSED ==> decoder [gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter0 [gstreamer] gstreamer stream status CREATE ==> src [gstreamer] gstreamer changed state from READY to PAUSED ==> v4l2src0 [gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline0 [gstreamer] gstreamer message new-clock ==> pipeline0 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter1 [gstreamer] gstreamer stream status ENTER ==> src [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> decoder [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter0 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> v4l2src0 21:37:53 | SUCCESS | VideoQuery - system ready * Serving Flask app 'nano_llm.web.server' * Debug mode: on 21:37:53 | INFO | WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. * Running on all addresses (0.0.0.0) * Running on https://127.0.0.1:8050 * Running on https://192.168.178.86:8050 21:37:53 | INFO | Press CTRL+C to quit [gstreamer] gstreamer message stream-start ==> pipeline0 [gstreamer] gstCamera -- onPreroll [gstreamer] gstBufferManager -- map buffer size was less than max size (1382400 vs 1382407) [gstreamer] gstBufferManager recieve caps: video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)1:4:0:0, framerate=(fraction)30/1 [gstreamer] gstBufferManager -- recieved first frame, codec=MJPEG format=i420 width=1280 height=720 size=1382407 [cuda] allocated 4 ring buffers (1382407 bytes each, 5529628 bytes total) [cuda] allocated 4 ring buffers (8 bytes each, 32 bytes total) [gstreamer] gstreamer changed state from READY to PAUSED ==> mysink [gstreamer] gstreamer message async-done ==> pipeline0 [gstreamer] gstreamer message latency ==> mysink [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> mysink [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> pipeline0 [cuda] allocated 4 ring buffers (2764800 bytes each, 11059200 bytes total) [cuda] allocated 2 ring buffers (1382400 bytes each, 2764800 bytes total) [gstreamer] gstEncoder -- starting pipeline, transitioning to GST_STATE_PLAYING [gstreamer] gstreamer changed state from NULL to READY ==> fakesink0 [gstreamer] gstreamer changed state from NULL to READY ==> queue0 [gstreamer] gstreamer changed state from NULL to READY ==> videotee [gstreamer] gstreamer changed state from NULL to READY ==> capsfilter3 [gstreamer] gstreamer changed state from NULL to READY ==> rtph264pay0 [gstreamer] gstreamer changed state from NULL to READY ==> capsfilter2 [gstreamer] gstreamer changed state from NULL to READY ==> encoder [gstreamer] gstreamer changed state from NULL to READY ==> mysource [gstreamer] gstreamer changed state from NULL to READY ==> pipeline1 [gstreamer] gstreamer stream status CREATE ==> src [gstreamer] gstreamer changed state from READY to PAUSED ==> queue0 [gstreamer] gstreamer changed state from READY to PAUSED ==> videotee [gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter3 [gstreamer] gstreamer changed state from READY to PAUSED ==> rtph264pay0 [gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter2 [gstreamer] gstreamer changed state from READY to PAUSED ==> encoder [gstreamer] gstreamer stream status CREATE ==> src [gstreamer] gstreamer changed state from READY to PAUSED ==> mysource [gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline1 [gstreamer] gstreamer message new-clock ==> pipeline1 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> queue0 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> videotee [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter3 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> rtph264pay0 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter2 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> encoder [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> mysource [gstreamer] gstreamer stream status ENTER ==> src [gstreamer] gstreamer stream status ENTER ==> src [gstreamer] gstEncoder -- new caps: video/x-raw, width=1280, height=720, format=(string)I420, framerate=30/1 [gstreamer] gstreamer message latency ==> encoder [gstreamer] gstreamer message stream-start ==> pipeline1 [gstreamer] gstreamer fakesink0 taglist, encoder=(string)x264, encoder-version=(uint)163, maximum-bitrate=(uint)4096000, nominal-bitrate=(uint)4096000; [gstreamer] gstreamer changed state from READY to PAUSED ==> fakesink0 [gstreamer] gstreamer message async-done ==> pipeline1 [gstreamer] gstreamer message latency ==> fakesink0 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> fakesink0 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> pipeline1 A white object is hanging from the ceiling. A white object is hanging from the ceiling. 21:37:57 | INFO | refresh rate: 0.73 FPS (1379.0 ms) A white object is hanging from the ceiling. 21:37:59 | INFO | refresh rate: 0.69 FPS (1455.4 ms) A white object is hanging from the ceiling. 21:38:00 | INFO | refresh rate: 0.69 FPS (1450.7 ms) A white object is hanging from the ceiling. 21:38:02 | INFO | refresh rate: 0.70 FPS (1422.5 ms) A white object is hanging from the ceiling. 21:38:03 | INFO | refresh rate: 0.69 FPS (1450.9 ms) A white object is hanging from the ceiling. 21:38:05 | INFO | refresh rate: 0.68 FPS (1477.4 ms) A white object is hanging from the ceiling. 21:38:06 | INFO | refresh rate: 0.68 FPS (1470.0 ms) A white object is hanging from the ceiling. 21:38:07 | INFO | refresh rate: 0.69 FPS (1448.5 ms) A white object is hanging from the ceiling. 21:38:09 | INFO | refresh rate: 0.69 FPS (1457.4 ms) A white object is hanging from the ceiling. 21:38:10 | INFO | refresh rate: 0.69 FPS (1439.7 ms) A white object is hanging from the ceiling. 21:38:12 | INFO | refresh rate: 0.68 FPS (1460.2 ms) A white object is hanging from the ceiling. 21:38:14 | INFO | refresh rate: 0.58 FPS (1726.3 ms) A white object is hanging from the ceiling. 21:38:15 | INFO | refresh rate: 0.70 FPS (1430.2 ms) A white object is hanging from the ceiling. 21:38:16 | INFO | refresh rate: 0.68 FPS (1472.3 ms) 21:38:17 | INFO | 127.0.0.1 - - [18/Jun/2025 21:38:17] "GET / HTTP/1.1" 200 - 21:38:17 | INFO | 127.0.0.1 - - [18/Jun/2025 21:38:17] "GET /static/select2.min.js HTTP/1.1" 200 - 21:38:17 | INFO | 127.0.0.1 - - [18/Jun/2025 21:38:17] "GET /static/select2.min.css HTTP/1.1" 200 - 21:38:17 | INFO | 127.0.0.1 - - [18/Jun/2025 21:38:17] "GET /static/chat.css HTTP/1.1" 200 - 21:38:17 | INFO | 127.0.0.1 - - [18/Jun/2025 21:38:17] "GET /static/bootstrap.css HTTP/1.1" 200 - 21:38:17 | INFO | 127.0.0.1 - - [18/Jun/2025 21:38:17] "GET /static/webrtc.js HTTP/1.1" 200 - 21:38:17 | INFO | 127.0.0.1 - - [18/Jun/2025 21:38:17] "GET /static/rest.js HTTP/1.1" 200 - 21:38:17 | INFO | 127.0.0.1 - - [18/Jun/2025 21:38:17] "GET /static/debounce.js HTTP/1.1" 200 - 21:38:17 | INFO | 127.0.0.1 - - [18/Jun/2025 21:38:17] "GET /static/websocket.js HTTP/1.1" 200 - 21:38:17 | INFO | 127.0.0.1 - - [18/Jun/2025 21:38:17] "GET /static/jquery-3.6.3.min.js HTTP/1.1" 200 - 21:38:17 | INFO | 127.0.0.1 - - [18/Jun/2025 21:38:17] "GET /static/bootstrap.bundle.min.js HTTP/1.1" 200 - A white object is hanging from the ceiling. [webrtc] websocket /output -- new connection opened by 127.0.0.1 (peer_id=0) [webrtc] new WebRTC peer connecting (127.0.0.1, peer_id=0) 21:38:18 | INFO | refresh rate: 0.56 FPS (1778.1 ms) ** ERROR:/opt/jetson-utils/codec/gstEncoder.cpp:876:static void gstEncoder::onWebsocketMessage(WebRTCPeer*, const char*, size_t, void*): 'sinkpad' should not be nullptr Bail out! ERROR:/opt/jetson-utils/codec/gstEncoder.cpp:876:static void gstEncoder::onWebsocketMessage(WebRTCPeer*, const char*, size_t, void*): 'sinkpad' should not be nullptr Fatal Python error: Aborted Thread 0x0000ffff563df120 (most recent call first): File "/usr/lib/python3.10/ssl.py", line 1371 in do_handshake File "/usr/local/lib/python3.10/dist-packages/websockets/sync/server.py", line 509 in conn_handler File "/usr/lib/python3.10/threading.py", line 953 in run File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap Thread 0x0000ffff58bef120 (most recent call first): File "/usr/lib/python3.10/selectors.py", line 416 in select File "/usr/lib/python3.10/socketserver.py", line 232 in serve_forever File "/usr/local/lib/python3.10/dist-packages/werkzeug/serving.py", line 817 in serve_forever File "/usr/local/lib/python3.10/dist-packages/werkzeug/serving.py", line 1123 in run_simple File "/usr/local/lib/python3.10/dist-packages/flask/app.py", line 625 in run File "/opt/NanoLLM/nano_llm/web/server.py", line 120 in <lambda> File "/usr/lib/python3.10/threading.py", line 953 in run File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap Thread 0x0000ffff5bc0f120 (most recent call first): File "/usr/lib/python3.10/selectors.py", line 469 in select File "/usr/local/lib/python3.10/dist-packages/websockets/sync/server.py", line 260 in serve_forever File "/opt/NanoLLM/nano_llm/web/server.py", line 119 in <lambda> File "/usr/lib/python3.10/threading.py", line 953 in run File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap Thread 0x0000ffff69c8f120 (most recent call first): File "/opt/NanoLLM/nano_llm/plugins/video/video_source.py", line 109 in capture File "/opt/NanoLLM/nano_llm/plugins/video/video_source.py", line 159 in run File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap Thread 0x0000ffff6747f120 (most recent call first): File "/opt/NanoLLM/nano_llm/agents/video_query.py", line 313 in poll_keyboard File "/usr/lib/python3.10/threading.py", line 953 in run File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap Thread 0x0000ffff64c6f120 (most recent call first): File "/usr/lib/python3.10/threading.py", line 324 in wait File "/usr/lib/python3.10/threading.py", line 607 in wait File "/opt/NanoLLM/nano_llm/plugin.py", line 335 in process_inputs File "/opt/NanoLLM/nano_llm/plugin.py", line 321 in run File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap Thread 0x0000fffe6d51f120 (most recent call first): File "/usr/lib/python3.10/threading.py", line 324 in wait File "/usr/lib/python3.10/threading.py", line 607 in wait File "/opt/NanoLLM/nano_llm/plugin.py", line 335 in process_inputs File "/opt/NanoLLM/nano_llm/plugin.py", line 321 in run File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap Thread 0x0000fffe6dd2f120 (most recent call first): File "/usr/lib/python3.10/threading.py", line 320 in wait File "/usr/lib/python3.10/threading.py", line 607 in wait File "/opt/NanoLLM/nano_llm/chat/stream.py", line 79 in __next__ File "/opt/NanoLLM/nano_llm/plugins/chat_query.py", line 223 in process File "/opt/NanoLLM/nano_llm/plugins/chat_query.py", line 152 in process File "/opt/NanoLLM/nano_llm/plugin.py", line 361 in dispatch File "/opt/NanoLLM/nano_llm/plugin.py", line 348 in process_inputs File "/opt/NanoLLM/nano_llm/plugin.py", line 321 in run File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap Thread 0x0000fffe9f6df120 (most recent call first): File "/opt/NanoLLM/nano_llm/models/mlc.py", line 436 in prefill File "/opt/NanoLLM/nano_llm/models/mlc.py", line 480 in _generate File "/opt/NanoLLM/nano_llm/models/mlc.py", line 538 in _run File "/usr/lib/python3.10/threading.py", line 953 in run File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap Thread 0x0000ffff14f7f120 (most recent call first): File "/usr/lib/python3.10/threading.py", line 324 in wait File "/usr/lib/python3.10/threading.py", line 607 in wait File "/usr/local/lib/python3.10/dist-packages/tqdm/_monitor.py", line 60 in run File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap Thread 0x0000ffffbf6866c0 (most recent call first): File "/usr/lib/python3.10/threading.py", line 1116 in _wait_for_tstate_lock File "/usr/lib/python3.10/threading.py", line 1096 in join File "/opt/NanoLLM/nano_llm/agent.py", line 58 in run File "/opt/NanoLLM/nano_llm/agents/video_query.py", line 357 in <module> File "/usr/lib/python3.10/runpy.py", line 86 in _run_code File "/usr/lib/python3.10/runpy.py", line 196 in _run_module_as_main Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, torch._C, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special[gstreamer] gstreamer webrtcbin-0 ERROR Your GStreamer installation is missing a plug-in. [gstreamer] gstreamer Debugging info: ../ext/webrtc/gstwebrtcbin.c(146): _have_nice_elements (): /GstPipeline:pipeline1/GstWebRTCBin:webrtcbin-0: libnice elements are not available , zstandard.backend_c, charset_normalizer.md21:38:18 | INFO | connection open , yaml._yaml21:38:18 | INFO | new websocket connection from ('127.0.0.1', 48848) 21:38:18 | INFO | listening on websocket connection from ('127.0.0.1', 48848) , sentencepiece._sentencepiece, psutil._psutil_linux, psutil._psutil_posix, PIL._imaging, PIL._imagingft, google.protobuf.pyext._message, jetson_utils_python, cuda._lib.utils, cuda._cuda.ccuda, cuda.ccuda, cuda.cuda, cuda._cuda.cnvrtc, cuda.cnvrtc, cuda.nvrtc, cuda._lib.ccudart.utils, cuda._lib.ccudart.ccudart, cuda.ccudart, cuda.cudart, _cffi_backend, pyaudio._portaudio, markupsafe._speedups, websockets.speedups, regex._regex, scipy._lib._ccallback_c, numba.core.typeconv._typeconv, numba._helperlib, numba._dynfunc, numba._dispatcher, numba.core.runtime._nrt_python, numba.np.ufunc._internal, numba.experimental.jitclass._box, h5py._errors, h5py.defs, h5py._objects, h5py.h5, h5py.utils, h5py.h5t, h5py.h5s, h5py.h5ac, h5py.h5p, h5py.h5r, h5py._proxy, h5py._conv, h5py.h5z, h5py.h5a, h5py.h5d, h5py.h5ds, h5py.h5g, h5py.h5i, h5py.h5f, h5py.h5fd, h5py.h5pl, h5py.h5o, h5py.h5l, h5py._selector, scipy.sparse._sparsetools, _csparsetools, scipy.sparse._csparsetools, scipy.linalg._fblas, scipy.linalg._flapack, scipy.linalg.cython_lapack, scipy.linalg._cythonized_array_utils, scipy.linalg._solve_toeplitz, scipy.linalg._flinalg, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_sqrtm_triu, scipy.linalg.cython_blas, scipy.linalg._matfuncs_expm, scipy.linalg._decomp_update, scipy.sparse.linalg._dsolve._superlu, scipy.sparse.linalg._eigen.arpack._arpack, scipy.sparse.csgraph._tools, scipy.sparse.csgraph._shortest_path, scipy.sparse.csgraph._traversal, scipy.sparse.csgraph._min_spanning_tree, scipy.sparse.csgraph._flow, scipy.sparse.csgraph._matching, scipy.sparse.csgraph._reordering, pyarrow.lib, pandas._libs.tslibs.ccalendar, pandas._libs.tslibs.np_datetime, pandas._libs.tslibs.dtypes, pandas._libs.tslibs.base, pandas._libs.tslibs.nattype, pandas._libs.tslibs.timezones, pandas._libs.tslibs.fields, pandas._libs.tslibs.timedeltas, pandas._libs.tslibs.tzconversion, pandas._libs.tslibs.timestamps, pandas._libs.properties, pandas._libs.tslibs.offsets, pandas._libs.tslibs.strptime, pandas._libs.tslibs.parsing, pandas._libs.tslibs.conversion, pandas._libs.tslibs.period, pandas._libs.tslibs.vectorized, pandas._libs.ops_dispatch, pandas._libs.missing, pandas._libs.hashtable, pandas._libs.algos, pandas._libs.interval, pandas._libs.lib, pyarrow._compute, pandas._libs.ops, numexpr.interpreter, pandas._libs.hashing, pandas._libs.arrays, pandas._libs.tslib, pandas._libs.sparse, pandas._libs.internals, pandas._libs.indexing, pandas._libs.index, pandas._libs.writers, pandas._libs.join, pandas._libs.window.aggregations, pandas._libs.window.indexers, pandas._libs.reshape, pandas._libs.groupby, pandas._libs.json21:38:18 | INFO | 127.0.0.1 - - [18/Jun/2025 21:38:18] "GET /static/favicon.ico HTTP/1.1" 200 - , pandas._libs.parsers, pandas._libs.testing, tvm._ffi._cy3.core (total: 145)