Skip to content

refactor: morphology per surface cluster - #3741

Draft
aclauer wants to merge 3 commits into
mainfrom
andrew/feat/morphology-mem-fix
Draft

refactor: morphology per surface cluster#3741
aclauer wants to merge 3 commits into
mainfrom
andrew/feat/morphology-mem-fix

Conversation

@aclauer

@aclauer aclauer commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator

Contribution path

  • Small, safe change that does not need a tracking issue
  • Linked issue or discussion: DIM-XXX / #XXX / URL

Problem

We use dilation + erosion for closing holes in the surfaces in the mls planner. We used imageproc for this, but that allocates an entire 2d "image" per z level, and if there are stray voxels far away causes massive images to get allocated.

Solution

  • hand write the morphology
  • collect clusters of surfaces that cannot interact with each other from the morphology, then operate on those

Massive benefits in really big spaces, and we can get rid of imageproc and all the deps that come from that.

How to Test

AI assistance

Checklist

  • I have read and approved the CLA.

@aclauer aclauer changed the title Swap imageproc for our own refactor: morphology per surface cluster Aug 27, 2026
@codecov

codecov Bot commented Aug 28, 2026

Copy link
Copy Markdown

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
4854 1 4853 87
View the top 1 failed test(s) by shortest run time
dimos.protocol.pubsub.shm.test_ipc_factory::test_concurrent_open_survives_the_creation_window
Stack Traces | 0.304s run time
slow_ftruncate = None

    def test_concurrent_open_survives_the_creation_window(slow_ftruncate) -> None:
        """Two peers opening the same segment at once: one creates, one waits it out.
    
        Before the readiness wait, the peer that lost the create raced into the
        creator's unsized segment and died with "cannot mmap an empty file".
        """
        tag = uuid.uuid4().hex[:12]
        data_name, ctrl_name = f"tq_{tag}_data", f"tq_{tag}_ctrl"
        channels: list[CpuShmQueue] = []
        errors: list[BaseException] = []
        lock = threading.Lock()
        barrier = threading.Barrier(2)
    
        def open_peer() -> None:
            barrier.wait()
            try:
                ch = CpuShmQueue((CAP,), np.uint8, data_name=data_name, ctrl_name=ctrl_name, slots=4)
            except BaseException as exc:
                with lock:
                    errors.append(exc)
                return
            with lock:
                channels.append(ch)
    
        threads = [threading.Thread(target=open_peer, daemon=True) for _ in range(2)]
        for t in threads:
            t.start()
        for t in threads:
            t.join(timeout=15)
    
        try:
>           assert not errors, f"concurrent open failed: {errors!r}"
E           AssertionError: concurrent open failed: [AssertionError("opened SHM 'tq_6555f35da2e4_data' is 0B < 352B needed: CpuShmQueue slots/capacity mismatch with the segment's creator")]
E           assert not [AssertionError("opened SHM 'tq_6555f35da2e4_data' is 0B < 352B needed: CpuShmQueue slots/capacity mismatch with the segment's creator")]

barrier    = <threading.Barrier at 0x7f34fff405c0: waiters=0/2>
ch         = <dimos.protocol.pubsub.shm.ipc_factory.CpuShmQueue object at 0x7f34fff40410>
channels   = [<dimos.protocol.pubsub.shm.ipc_factory.CpuShmQueue object at 0x7f34fff40410>]
ctrl_name  = 'tq_6555f35da2e4_ctrl'
data_name  = 'tq_6555f35da2e4_data'
errors     = [AssertionError("opened SHM 'tq_6555f35da2e4_data' is 0B < 352B needed: CpuShmQueue slots/capacity mismatch with the segment's creator")]
lock       = <unlocked _thread.lock object at 0x7f34ffe1bd80>
open_peer  = <function test_concurrent_open_survives_the_creation_window.<locals>.open_peer at 0x7f34ffff4220>
slow_ftruncate = None
t          = <Thread(Thread-2810 (open_peer), stopped daemon 139865572239040)>
tag        = '6555f35da2e4'
threads    = [<Thread(Thread-2809 (open_peer), stopped daemon 139862170511040)>, <Thread(Thread-2810 (open_peer), stopped daemon 139865572239040)>]

.../pubsub/shm/test_ipc_factory.py:235: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant