Incrementally Tuning Synthetic Training Datasets for Satellite Object Detection

Last year I had the honor of partnering with Phillip Hale to work on a synthetic data whitepaper and am thrilled to share our results. I was responsible for all data generation. I iterated with Phillip to test variables and refine the data generation pipeline. 

I have found that iteration is often undervalued when it comes to synthetic data. It is often much more effective to take a small team and a more reduced approach to data generation but iterate daily, than to build a complex pipeline with a large team with one month turn arounds.

You can read the full paper here:

OnlyPlanes/OnlyPlanes.pdf at master · naivelogic/OnlyPlanes (github.com)

How to use PDG services to speed up PDG ropfetches in less than 5 minutes

Why do I need this? basically by default Houdini will boot up a new hython instance for every work item as it cooks in a rop fetch. this boot up period takes some time and if the export is faily simple, then booting hython might be the part of completing that work item that takes the longest. Using PDG services has brought my completion times in some cases from 3+ hours down to five minutes. (I tend to do mass exports of variations of simple geometry using PDG to generate these variations)

All PDG services does is leave several instances of hython running in the background so that they are always ready to process new work items without ever closing and rebooting.

Here is the quick start:

Given a PDG layout similar to this select your ropfetch

Select the Service tab

Toggle use ROP Fetch service

Open the tasks menu

Open PDG services

Add a service

Set the pool size to however many work items you want to run at the same time then click add.

Click start to start up the instances and stop to stop them. Any time you cook the ROP fetch while it is running it will default to using PDG services.

Now when you run your ropfetch it will not reboot hython every single work item. Congrats you just saved yourself a lot of time.

Running Bifrost Compounds through Houdini PDG

Running Bifrost compounds through Houdini PDG using bifcmd while passing in attributes dynamically is surprisingly simple

bifcmd is an executable that Bifrost provides to run Bifrost compounds headless through the command line.

STEP 1: build and publish a compound with no auto ports

For demo purpose I’ve made a simple Bifrost compound with a seed, index, and file_path input exposed this compound makes a cube randomly with a height between 1 and 2 and saves it to the file out location.

It looks like this inside

The cube looks like this

I will now publish the compound

Copy the namespace somewhere we will need that later as well as the path to the node save location

Mine Namespace is: User::Bifcmd::rand_cube_height

STEP 2: Set up environment in Houdini

Open Houdini

Create a top network

Dive inside

Select the local scheduler

Open the “Job Parms” panel

Scroll to the bottom and add an environment variable and name it PATH

We need to add the 3 folder paths to this Environment variable

the folder which contains bifcmd.exe (bifcmd.exe will launch a headless version of Bifrost):

C:\Program Files\Autodesk\Bifrost\Maya2023\2.6.0.0\bifrost\bin

The Bifrost bifcmd dependency location:

C:\Program Files\Autodesk\Bifrost\Maya2023\2.6.0.0\bifrost\thirdparty\bin

And the install location for BifrostUSD:

C:\Program Files\Autodesk\Bifrost\Maya2023\2.6.0.0\bifrost\packs\usd_pack\0.21.11\thirdparty\usd-0.21.11\bin

The locations might be different on your computer

Join the locations with a semicolon:

C:\Program Files\Autodesk\Bifrost\Maya2023\2.6.0.0\bifrost\packs\usd_pack\0.21.11\thirdparty\usd-0.21.11\bin;C:\Program Files\Autodesk\Bifrost\Maya2023\2.6.0.0\bifrost\thirdparty\bin;C:\Program Files\Autodesk\Bifrost\Maya2023\2.6.0.0\bifrost\bin

And paste them into the value box

STEP 3: add PDG attributes and shell command

Add a wedge node with count 10 and an integer attribute called seed

Set Seeds wedge type to value and the number to 0

Now place a generic generator node and connect it

In the generic generator check the run command in system shell button

Now we need to build the command

bifcmd C:\PATH_TO\rand_cube_height.json

this is the base command that tell bifcmd what json to use to run the node this should be the path to the published compound we made earlier

 

--set-port seed `@seed`

This flag will set given port to a given value. In this case the value is the PDG attribute @seed that we are passing in

 

--set-port index `@wedgeindex`

This flag will set given port to a given value. In this case the value is the PDG attribute @wedgeindex that we are passing in

 

 --set-port file_out C:\Proj\BifrostTools\BifCmd\output\cube`@wedgeindex`.usd 

This flag will set given port to a given value. In this case the value is the save path and we are using the PDG attribute @wedgeindex to make sure every iteration has a unique name. pick somewhere on your computer where you would like to save this

 

--bifrost-location "C:\Program Files\Autodesk\Bifrost\Maya2023\2.6.0.0\bifrost"

This flag tells bifcmd where bifrost is installed

 

--compound-name "User::Bifcmd::rand_cube_height"

This flag tells bifcmd the namespace of the node we will run, this is important because the json we pointed to earlier could have multiple nodes in it

 

The final command should look something like this:

bifcmd C:\PATH_TO\rand_cube_height.json --set-port seed `@seed` --set-port index `@wedgeindex` --set-port file_out C:\Proj\BifrostTools\BifCmd\output\cube`@wedgeindex`.usd  --bifrost-location "C:\Program Files\Autodesk\Bifrost\Maya2023\2.6.0.0\bifrost" --compound-name "User::Bifcmd::rand_cube_height"

 

 Put your command in the generic generator

Select the generic generator and hit SHIFT- V

your output folder should now have 10 cubes

If you open each cube each one should be a different height

That’s it. You have now run a Bifrost node in Houdini