You must start by going here to your Business Settings and turn on "SKU Level Inventory Source". If you do not have access to this feature, that means you are not on the correct level Plan and will need to upgrade to gain access to this feature.
Once enabled, you will go to Inventory>Manage Inventory, where you will see a new option for SKU Level Inventory Source.
Clicking on that will show you the inventory sources that are enabled for each SKU and marketplace.
Please note, this feature is for "opting out" a Market SKU's warehouse. So you would want to enable the desired inventory source for your marketplace at the integration level, then come to this page and turn off all the SKUs you do not want to sell that particular inventory source.
You can do so in bulk by checking the boxes to the left of the Product, then click on "Bulk Update (x)".
Then select the inventory source / warehouse you want to include/exclude, then select the marketplace integration and click Update Selected Items.