Some notes from discussion with @sosiouxme

oo-gear-firewall can be run prior to districting but should be re-run after any changes to UID ranges.

oo-gear-firewall is safe to run repeatedly, however selinux module manipulation is a slow operation and it creates a window of opportunity when the policy is removed and then re-applied. In my testing it took about 90 seconds to remove and re-apply the policy. We don't want to do that every 30 minutes.
Therefore, we should ensure that we only run these commands when necessary. At first I thought we could get away with only doing this if rhc-app-contain rule didn't exist or if the UID ranges have been modified, but I think that's a significant deviation from what most people expect from puppet. People expect puppet to protect against outside manipulation or to fail.

With that in mind I think we should write a wrapper to ensure that proposed state is a subset of current state for both iptables and selinux and if so exit with return code 0 otherwise execute the desired commands. Ideally this could be implemented in oo-gear-firewall at some point.

One thing to keep in mind is how our use of lokkit in the puppet module would affect running oo-gear-firewall.

I see a few options here:

1. re-write the logic in oo-gear-firewall in puppet (potentially a maintenance nightmare)
2. use the output mode (or potentially dry-run for iptables) of oo-gear-firewall to generate the puppet rules. This might require changes to oo-gear-firewall to potentially allow an output that is more consumable.
3. if there are conflicts with lokkit and oo-gear-firewall, then convert the usage of puppet firewall rules to not use lokkit (the lokkit module is currently used because of performance issues when using the firewall module and large rulesets, since rules are applied one at a time)

Another thing to note is that currently origin milestone 4 does not have oo-gear-firewall, so we would need to take that into account if using the oo-gear-firewall binary.

lokkit currently overrides the oo-gear-firewall iptable rules.

Using dry-run we are left with a file at /etc/sysconfig/iptables.contained it wouldn't be the "ideal" way which puppet should run, but perhaps a shortcut could be to use that file as a flag to prevent lokkit from re-running. Short of rewriting to use puppet-firewall...

There also seems to be a conflicting sysctl value, here's the final report:

/Stage[main]/Openshift_origin::Node/Sysctl::Value[net.netfilter.nf_conntrack_max]/Exec[exec_sysctl_net.netfilter.nf_conntrack_max]/returns executed successfully
/Stage[main]/Openshift_origin::Node/Exec[prevent_node_frontend_changes]/returns executed successfully
/Stage[main]/Lokkit/Exec[lokkit_update]/returns executed successfully

Perhaps for iptables, there appears to be a "custom" rule set option available for the lokkit module [1]

Would it be worth while considering having lokkit source the dryrun rules? Dryrun also has the other rules as well so would need to be filtered - just tossing around a possible idea.

[1] https://forge.puppetlabs.com/rharrison/lokkit#examples-1