Freeside:3:Documentation:Developer:Billing Internals
FS::cust_main::Billing::bill
This is the main method for generating invoices. It's called on a single FS::cust_main, finds any packages that are due to be billed, and creates one or more invoices with the package charges.
It does not handle sending the invoice to the customer, charging their credit card, suspending packages if they're overdue, or anything else of that kind. Those are events run from collect().
Initial setup
Grab the customer and options. Immediately exit if the customer is “complimentary”. Set the debug level, create a log context, and write a log entry.
my( $self, %options ) = @_;
return if $self->complimentary eq 'Y';
local($DEBUG) = $FS::cust_main::DEBUG if $FS::cust_main::DEBUG > $DEBUG; my $log = FS::Log->new('FS::cust_main::Billing::bill'); my %logopt = (object => $self);
$log->debug('start', %logopt); warn "$me bill customer ". $self->custnum. "\n" if $DEBUG;
Set the billing clock. $time is the date we are “billing on”. For example, freeside-daily -y (bill packages a few days in the future so that invoices can be mailed out early) sets $time. This is almost, but not exactly, the same as changing the system clock.
$invoice_time is the date that will appear on invoices. Normally it's the same as $time; the -n option overrides that and forces it to be the real system time.
$cmp_time is the “comparison time”: a package gets billed if its next-bill date is <= $cmp_time. This is the same as $time unless next-bill-ignore-time is set, in which case we use the end of the day. Note that this does not affect certain other important time values such as prorate cutoffs.
Also parse the 'not_pkgpart' option to exclude specific packages from billing.
my $time = $options{'time'} || time; my $invoice_time = $options{'invoice_time'} || $time;
my $cmp_time = ( $conf->exists('next-bill-ignore-time') ? day_end( $time ) : $time );
$options{'not_pkgpart'} ||= {}; $options{'not_pkgpart'} = { map { $_ => 1 } split(/\s*,\s*/, $options{'not_pkgpart'}) } unless ref($options{'not_pkgpart'});
Block out interruptions, start a transaction, and lock the customer record (so that packages aren't ordered or status-changed, payments aren't recorded, etc. while doing this). If anything goes wrong during billing, we will rollback and return an error.
local $SIG{HUP} = 'IGNORE'; local $SIG{INT} = 'IGNORE'; local $SIG{QUIT} = 'IGNORE'; local $SIG{TERM} = 'IGNORE'; local $SIG{TSTP} = 'IGNORE'; local $SIG{PIPE} = 'IGNORE';
my $oldAutoCommit = $FS::UID::AutoCommit; local $FS::UID::AutoCommit = 0; my $dbh = dbh;
$log->debug('acquiring lock', %logopt); warn "$me acquiring lock on customer ". $self->custnum. "\n" if $DEBUG;
$self->select_for_update; #mutex
Run pre-billing events.
Certain billing event actions are always run just before billing, because they affect the charges that will appear on the bill. These are:
- pkg_discount
- cust_pkg_fee, cust_fee, and pkg_fee
- The old “fee” event that created fees as one-time charges.
In this section we call do_cust_event() with a 'stage' parameter to specify only the pre-billing events.
$log->debug('running pre-bill events', %logopt); warn "$me running pre-bill events for customer ". $self->custnum. "\n" if $DEBUG;
my $error = $self->do_cust_event( 'debug' => ( $options{'debug'} || 0 ), 'time' => $invoice_time, 'check_freq' => $options{'check_freq'}, 'stage' => 'pre-bill', ) unless $options{no_commit}; if ( $error ) { $dbh->rollback if $oldAutoCommit && !$options{no_commit}; return $error; }
$log->debug('done running pre-bill events', %logopt); warn "$me done running pre-bill events for customer ". $self->custnum. "\n" if $DEBUG;
Identify billable packages and add-on packages.
Set up space for the line items (%cust_bill_pkg), setup and recur totals, and a tax engine. Every time we add a line item to the invoice, we must push it to the line item array, add its setup and recur charges to the totals, and send it to the tax engine.
A single call to bill() can generate multiple invoices. Packages can be eligible or ineligible for automatic payment; that's implemented by splitting non-auto-payment packages off onto a separate invoice. So we'll (potentially) make two invoices here, and need a line items array, subtotals, etc. for each of them. If any package has the “separate_bill” flag, we'll make an additional invoice just for that package.
Also, if the set of packages to bill wasn't specified ('pkg_list'; it's usually not), then use all of the customer's non-canceled packages. We will check later for whether they're due for billing; that's complicated.
#keep auto-charge and non-auto-charge line items separate my @passes = ( , 'no_auto' );
my %cust_bill_pkg = map { $_ => [] } @passes;
### # find the packages which are due for billing, find out how much they are # & generate invoice database. ###
my %total_setup = map { my $z = 0; $_ => \$z; } @passes; my %total_recur = map { my $z = 0; $_ => \$z; } @passes;
my @precommit_hooks = ();
$options{'pkg_list'} ||= [ $self->ncancelled_pkgs ]; #param checks?
my %tax_engines; my $tax_is_batch = ; foreach (@passes) { $tax_engines{$_} = FS::TaxEngine->new(cust_main => $self, invoice_time => $invoice_time, cancel => $options{cancel}, estimate => $options{estimate}, ); $tax_is_batch ||= $tax_engines{$_}->info->{batch}; }
Here we start the main loop over all packages. Skip any that are excluded by 'not_pkgpart', or by 'no_prepaid'. (freeside-daily sets this, as prepaid packages are billed by another mechanism.) And if there's an explicit 'pkg_list' then use only those.
foreach my $cust_pkg ( @{ $options{'pkg_list'} } ) {
next if $options{'not_pkgpart'}->{$cust_pkg->pkgpart};
my $part_pkg = $cust_pkg->part_pkg;
next if $options{'no_prepaid'} && $part_pkg->is_prepaid;
$log->debug('bill package '. $cust_pkg->pkgnum, %logopt); warn " bill package ". $cust_pkg->pkgnum. "\n" if $DEBUG;
#? to avoid use of uninitialized value errors... ? $cust_pkg->setfield('bill', ) unless defined($cust_pkg->bill);
Call self_and_bill_linked() to find any billing add-on packages. Add-ons are additional package definitions (part_pkg records, plus all their accessories) that are linked to a “main” package definition, and generate additional charges on the bill. For example, a phone package that has a monthly fee plus long-distance usage billing would have an add-on voip_cdr package to do the usage billing.
Add-on packages always create separate cust_bill_pkg records, with the “pkgpart_override” field referencing the add-on package. Normally, these charges will show separately on the invoice, but they can be “bundled” with the main package by setting part_pkg_link.hidden. At this point we set a flag if this looks like a bundle package, as that will matter later.
my $real_pkgpart = $cust_pkg->pkgpart; my %hash = $cust_pkg->hash;
# we could implement this bit as FS::part_pkg::has_hidden, but we already # suffer from performance issues $options{has_hidden} = 0; my @part_pkg = $part_pkg->self_and_bill_linked; $options{has_hidden} = 1 if ($part_pkg[1] && $part_pkg[1]->hidden);
Create line items for each billable package.
As we create line items we'll push them onto $cust_bill_pkg{$pass} (either or 'no_auto'). First, balance transfers: if this package is being billed for the first time after a package change, and package balances are enabled, then credit the old package to zero out its balance, and make a line item to carry the balance forward to its successor package.
# if this package was changed from another package, # and it hasn't been billed since then, # and package balances are enabled, if ( $cust_pkg->change_pkgnum and $cust_pkg->change_date >= ($cust_pkg->last_bill || 0) and $cust_pkg->change_date < $invoice_time and $conf->exists('pkg-balances') ) { # _transfer_balance will also create the appropriate credit my @transfer_items = $self->_transfer_balance($cust_pkg); # $part_pkg[0] is the "real" part_pkg my $pass = ($cust_pkg->no_auto || $part_pkg[0]->no_auto) ? 'no_auto' : ; push @{ $cust_bill_pkg{$pass} }, @transfer_items; # treating this as recur, just because most charges are recur... ${$total_recur{$pass}} += $_->recur foreach @transfer_items;
# currently not considering separate_bill here, as it's for # one-time charges only }
Loop over the part_pkgs (first the “real” package definition, then the add-ons) and decide which invoice to put the charges on. Normally, $pass = , which will put all charges on a single invoice, but no_auto packages have to go on another invoice, and each package with separate_bill needs yet another (so it will create entries in %total_setup, %total_recur, %cust_bill_pkg, etc.).
foreach my $part_pkg ( @part_pkg ) {
my $this_cust_pkg = $cust_pkg; # for add-on packages, copy the object to avoid leaking changes back to # the caller if pkg_list is in use; see RT#73607 if ( $part_pkg->get('pkgpart') != $real_pkgpart ) { $this_cust_pkg = FS::cust_pkg->new({ %hash }); }
my $pass = ; if ( $this_cust_pkg->separate_bill ) { # if no_auto is also set, that's fine. we just need to not have # invoices that are both auto and no_auto, and since the package # gets an invoice all to itself, it will only be one or the other. $pass = $this_cust_pkg->pkgnum; if (!exists $cust_bill_pkg{$pass}) { # it may not exist yet push @passes, $pass; $total_setup{$pass} = do { my $z = 0; \$z }; $total_recur{$pass} = do { my $z = 0; \$z }; # it also needs its own tax context $tax_engines{$pass} = FS::TaxEngine->new( cust_main => $self, invoice_time => $invoice_time, cancel => $options{cancel}, estimate => $options{estimate}, ); $cust_bill_pkg{$pass} = []; } } elsif ( ($this_cust_pkg->no_auto || $part_pkg->no_auto) ) { $pass = 'no_auto'; }
The loop here checks whether the package is due for billing, and handles back-billing of multiple cycles. For example, if a monthly package's bill date is more than a month in the past, we'll bill as many months as necessary to bring the package up to date. The loop normally exits once the package bill date is in the future, but will exit after a single pass if we're billing on cancellation, or if freeside-daily -o is in use. It will also exit if there's an error or if the bill date isn't being incremented (as for a one-time charge).
_make_lines() is complex enough to have [#_make_lines its own section of the docs], but it generates a line item and does the following with it:
- appends it to the arrayref in 'line_items' (so, to $cust_bill_pkg{$pass}).
- adds its setup and recur fees to the running totals
- informs the tax engine about it (via the add_sale() method)
- increments the package's last-bill and next-bill dates
- returns an error if anything goes wrong
my $next_bill = $this_cust_pkg->getfield('bill') || 0; my $error; # let this run once if this is the last bill upon cancellation while ( $next_bill <= $cmp_time or $options{cancel} ) { $error = $self->_make_lines( 'part_pkg' => $part_pkg, 'cust_pkg' => $this_cust_pkg, 'precommit_hooks' => \@precommit_hooks, 'line_items' => $cust_bill_pkg{$pass}, 'setup' => $total_setup{$pass}, 'recur' => $total_recur{$pass}, 'tax_engine' => $tax_engines{$pass}, 'time' => $time, 'real_pkgpart' => $real_pkgpart, 'options' => \%options, );
# Stop if anything goes wrong last if $error;
# or if we're not incrementing the bill date. last if ($this_cust_pkg->getfield('bill') || 0) == $next_bill;
# or if we're letting it run only once last if $options{cancel};
$next_bill = $this_cust_pkg->getfield('bill') || 0;
#stop if -o was passed to freeside-daily last if $options{'one_recur'}; } if ($error) { $dbh->rollback if $oldAutoCommit && !$options{no_commit}; return $error; }
} #foreach my $part_pkg
} #foreach my $cust_pkg
Identify applicable fees and tax adjustments and create line items.
At this point we have a list of package line items for each invoice the customer might receive. Now loop through the (possible) invoices and process non-package fees.
A “fee origin” (FS::FeeOrigin_Mixin) is an instruction or “order” to charge a fee; the classes are FS::cust_event_fee (fees generated by billing events, such as late fees) and FS::cust_pkg_reason_fee (unsuspension fees). When the fee is put on an invoice, the order is fulfilled; the billpkgnum of the fee line item is then recorded on the fee origin record.
The by_cust() method does a search for all fee origins for a customer; we limit it to records with billpkgnum = NULL to get fees that still need to be charged. Those are @pending_fees.
We process fees at this stage (after generating all package line items) because we need to know if the customer is going to receive an invoice or not. Fees can be defined (via the 'nextbill' flag) to be charged immediately, or on the customer's next bill. If all pending fees have 'nextbill' and there aren't any package charges on this invoice (@cust_bill_pkg is empty) then skip to the next invoice. If the customer has no package charges at all, then the 'nextbill' fees won't be charged at all on this billing date and will remain pending.
foreach my $pass (@passes) { # keys %cust_bill_pkg )
my @cust_bill_pkg = _omit_zero_value_bundles(@{ $cust_bill_pkg{$pass} });
warn "$me billing pass $pass\n" #.Dumper(\@cust_bill_pkg)."\n" if $DEBUG > 2;
### # process fees ###
my @pending_fees = FS::FeeOrigin_Mixin->by_cust($self->custnum, hashref => { 'billpkgnum' => } ); warn "$me found pending fees:\n".Dumper(\@pending_fees)."\n" if @pending_fees and $DEBUG > 1;
# determine whether to generate an invoice my $generate_bill = scalar(@cust_bill_pkg) > 0;
foreach my $fee (@pending_fees) { $generate_bill = 1 unless $fee->nextbill; }
# don't create an invoice with no line items, or where the only line # items are fees that are supposed to be held until the next invoice next if !$generate_bill;
For each fee origin, do some things. Skip the fee if the fee definition is somehow inappropriate:
# calculate fees... my @fee_items; foreach my $fee_origin (@pending_fees) { my $part_fee = $fee_origin->part_fee;
# check whether the fee is applicable before doing anything expensive: # # if the fee def belongs to a different agent, don't charge the fee. # event conditions should prevent this, but just in case they don't, # skip the fee. if ( $part_fee->agentnum and $part_fee->agentnum != $self->agentnum ) { warn "tried to charge fee#".$part_fee->feepart . " on customer#".$self->custnum." from a different agent.\n"; next; } # also skip if it's disabled next if $part_fee->disabled eq 'Y';
Then determine the “basis” for the fee, in case it's defined as a percentage of something. The fee origin may declare that the fee applies to a previous invoice (such as a finance charge). Otherwise, the fee applies to the current invoice. Since the current invoice doesn't exist yet, construct one temporarily that has the line items of the current invoice.
# Decide which invoice to base the fee on. my $cust_bill = $fee_origin->cust_bill; if (!$cust_bill) { # Then link it to the current invoice. This isn't the real cust_bill # object that will be inserted--in particular there are no taxes yet. # If you want to charge a fee on the total invoice amount including # taxes, you have to put the fee on the next invoice. $cust_bill = FS::cust_bill->new({ 'custnum' => $self->custnum, 'cust_bill_pkg' => \@cust_bill_pkg, 'charged' => ${ $total_setup{$pass} } + ${ $total_recur{$pass} }, }); # If the origin is for a specific package, then only apply the fee to # line items from that package. if ( my $cust_pkg = $fee_origin->cust_pkg ) { my @charge_fee_on_item; my $charge_fee_on_amount = 0; foreach (@cust_bill_pkg) { if ($_->pkgnum == $cust_pkg->pkgnum) { push @charge_fee_on_item, $_; $charge_fee_on_amount += $_->setup + $_->recur; } } $cust_bill->set('cust_bill_pkg', \@charge_fee_on_item); $cust_bill->set('charged', $charge_fee_on_amount); }
} # $cust_bill is now set
FS::part_fee->lineitem() asks the fee definition to make a cust_bill_pkg record for this fee, as applied to a specific invoice. After the invoice is inserted, we're going to need to record the billpkgnum on the fee origin record. FS::cust_bill_pkg->insert knows to do this.
Then append any fee line items to the invoice, add them to the running totals, and send them to the tax engine.
# calculate the fee my $fee_item = $part_fee->lineitem($cust_bill) or next; # link this so that we can clear the marker on inserting the line item $fee_item->set('fee_origin', $fee_origin); push @fee_items, $fee_item;
}
# add fees to the invoice foreach my $fee_item (@fee_items) {
push @cust_bill_pkg, $fee_item; ${ $total_setup{$pass} } += $fee_item->setup; ${ $total_recur{$pass} } += $fee_item->recur;
my $part_fee = $fee_item->part_fee; my $fee_location = $self->ship_location; # I think?
my $error = $tax_engines{}->add_sale($fee_item);
return $error if $error;
}
Add the special “postal invoice fee”. This was a per-invoice flat fee for sending customers a printed invoice; like all “old” fees it worked by adding a one-time charge. charge_postal_fee() is the method to do that. It adds a one-time package to the customer in the middle of billing, after all their other packages have been billed, for exactly the same reason that other fees are processed at this point in billing. This feature is obsolete so I won't say any more.
# XXX implementation of fees is supposed to make this go away... if ( scalar( grep { $_->recur && $_->recur > 0 } @cust_bill_pkg) || !$conf->exists('postal_invoice-recurring_only') ) {
my $postal_pkg = $self->charge_postal_fee(); if ( $postal_pkg && !ref( $postal_pkg ) ) {
$dbh->rollback if $oldAutoCommit && !$options{no_commit}; return "can't charge postal invoice fee for customer ". $self->custnum. ": $postal_pkg";
} elsif ( $postal_pkg ) {
my $real_pkgpart = $postal_pkg->pkgpart; # we could implement this bit as FS::part_pkg::has_hidden, but we already # suffer from performance issues $options{has_hidden} = 0; my @part_pkg = $postal_pkg->part_pkg->self_and_bill_linked; $options{has_hidden} = 1 if ($part_pkg[1] && $part_pkg[1]->hidden);
foreach my $part_pkg ( @part_pkg ) { my %postal_options = %options; delete $postal_options{cancel}; my $error = $self->_make_lines( 'part_pkg' => $part_pkg, 'cust_pkg' => $postal_pkg, 'precommit_hooks' => \@precommit_hooks, 'line_items' => \@cust_bill_pkg, 'setup' => $total_setup{$pass}, 'recur' => $total_recur{$pass}, 'tax_engine' => $tax_engines{$pass}, 'time' => $time, 'real_pkgpart' => $real_pkgpart, 'options' => \%postal_options, ); if ($error) { $dbh->rollback if $oldAutoCommit && !$options{no_commit}; return $error; } }
# it's silly to have a zero value postal_pkg, but.... @cust_bill_pkg = _omit_zero_value_bundles(@cust_bill_pkg);
}
}
Tax adjustments (FS::cust_tax_adjustment) are manual adjustment records that work much like fee origins, but are created manually. If a tax adjustment exists for the customer, and doesn't have a billpkgnum yet, and an invoice is being generated, then create a line item with the amount and itemdesc specified in the tax adjustment. Note that this line item is a tax line item: it has pkgnum = 0, feepart = NULL, and recur = 0. On that line item, set 'cust_tax_adjustment' to the tax adjustment record so that cust_bill_pkg->insert can record that the adjustment was billed.
#add tax adjustments #XXX does this work with batch tax engines? warn "adding tax adjustments...\n" if $DEBUG > 2; foreach my $cust_tax_adjustment ( qsearch('cust_tax_adjustment', { 'custnum' => $self->custnum, 'billpkgnum' => , } ) ) {
my $tax = sprintf('%.2f', $cust_tax_adjustment->amount );
my $itemdesc = $cust_tax_adjustment->taxname; $itemdesc = if $itemdesc eq 'Tax';
push @cust_bill_pkg, new FS::cust_bill_pkg { 'pkgnum' => 0, 'setup' => $tax, 'recur' => 0, 'sdate' => '', 'edate' => '', 'itemdesc' => $itemdesc, 'itemcomment' => $cust_tax_adjustment->comment, 'cust_tax_adjustment' => $cust_tax_adjustment, #'cust_bill_pkg_tax_location' => \@cust_bill_pkg_tax_location, };
}
Create the invoice record.
Before storing the record, calculate some totals:
- cust_bill.charged: the sum of charges on this invoice (taxes will be added later)
- billing_balance: the customer's balance before this invoice.
- previous_balance: the customer's balance immediately after their previous invoice, determined as (billing_balance + charged) on that invoice.
Note that billing_balance and previous_balance are obsolete. They were historically used to create invoice summary sections (“Balance on previous invoice”, “Payments/credits since then”), but since 2014 we actually look at the transaction history. This method produces correct results if transactions get voided.
my $charged = sprintf('%.2f', ${ $total_setup{$pass} } + ${ $total_recur{$pass} } );
my $balance = $self->balance;
my $previous_bill = qsearchs({ 'table' => 'cust_bill', 'hashref' => { custnum=>$self->custnum }, 'extra_sql' => 'ORDER BY _date DESC LIMIT 1', }); my $previous_balance = $previous_bill ? ( $previous_bill->billing_balance + $previous_bill->charged ) : 0;
Now store the record. FS::cust_bill::insert() will notice the arrayref of line items and insert all of them. Line items with 'fee_origin' or 'tax_adjustment' references will also trigger updates to those records.
The 'no_commit' option is for simulating billing, and will stop the record from being inserted. Currently this is used only with term prepayment discounts (which are no longer a supported feature) to figure out the “amount saved” over the prepaid term.
Note that (like almost anywhere else in Freeside) $FS::UID::AutoCommit = 0 will also prevent bill() from committing its changes. This is how quotations work: FS::quotation->estimate opens a transaction, inserts packages, runs bill(), and then reverts the whole transaction.
$log->debug('creating the new invoice', %logopt); warn "creating the new invoice\n" if $DEBUG; #create the new invoice my $cust_bill = new FS::cust_bill ( { 'custnum' => $self->custnum, '_date' => $invoice_time, 'charged' => $charged, 'billing_balance' => $balance, 'previous_balance' => $previous_balance, 'invoice_terms' => $options{'invoice_terms'}, 'cust_bill_pkg' => \@cust_bill_pkg, 'pending' => 'Y', # clear this after doing taxes } );
if (!$options{no_commit}) { # probably we ought to insert it as pending, and then rollback # without ever un-pending it $error = $cust_bill->insert; if ( $error ) { $dbh->rollback if $oldAutoCommit && !$options{no_commit}; return "can't create invoice for customer #". $self->custnum. ": $error"; }
}
Calculate taxes and insert tax line items.
$tax_is_batch was previously set to true if we're using a batch tax processor. In that case, don't do any of this; just leave the invoice flagged as pending and FS::Cron::tax_batch will do the rest.
Otherwise: get the tax engine we created for this invoice pass, and call calculate_taxes(), passing the pending invoice. This will return an arrayref of line items, or throw an exception. If it throws an exception, rollback and exit.
# calculate and append taxes if ( ! $tax_is_batch) { local $@; my $arrayref = eval { $tax_engines{$pass}->calculate_taxes($cust_bill) };
if ( $@ ) { $dbh->rollback if $oldAutoCommit && !$options{no_commit}; return $@; }
Append each tax line item to the invoice (directly, by setting invnum and inserting them) and add the charges to a running total. Tax line items have 'setup' but not 'recur' charges.
# or should this be in TaxEngine? my $total_tax = 0; foreach my $taxline ( @$arrayref ) { $total_tax += $taxline->setup; $taxline->set('invnum' => $cust_bill->invnum); # just to be sure push @cust_bill_pkg, $taxline; # for return_bill
if (!$options{no_commit}) { my $error = $taxline->insert; if ( $error ) { $dbh->rollback if $oldAutoCommit; return $error; } }
}
Add the total tax to the total invoice charges, remove the pending flag, and then update the invoice record. Then we're done.
# add tax to the invoice amount and finalize it ${ $total_setup{$pass} } = sprintf('%.2f', ${ $total_setup{$pass} } + $total_tax); $charged = sprintf('%.2f', $charged + $total_tax); $cust_bill->set('charged', $charged); $cust_bill->set('pending', );
if (!$options{no_commit}) { my $error = $cust_bill->replace; if ( $error ) { $dbh->rollback if $oldAutoCommit; return $error; } }
} # if !$tax_is_batch # if it IS batch, then we'll do all this in process_tax_batch
After dealing with this invoice, push it to the 'return_bill' arrayref if there is one. This will allow the caller to see which invoices were created if they want.
push @{$options{return_bill}}, $cust_bill if $options{return_bill};
} #foreach my $pass ( keys %cust_bill_pkg )
Run any post-invoicing pre-commit hooks. These are currently used for only one thing: for packages with time-limited discounts, we need to increment the number of discount months used after completely billing the package, but before doing anything else. FS::part_pkg::discount_Mixin sets this up.
Then commit changes if needed, and return.
foreach my $hook ( @precommit_hooks ) { eval { &{$hook}; #($self) ? } unless $options{no_commit}; if ( $@ ) { $dbh->rollback if $oldAutoCommit && !$options{no_commit}; return "$@ running precommit hook $hook\n"; } }
$dbh->commit or die $dbh->errstr if $oldAutoCommit && !$options{no_commit};
; #no error }
FS::cust_main::Billing::_make_lines
This is the other most important billing method. It takes a single package and package definition, and creates at most one line item for that package. This is called only from bill() and should probably never be used any other way, since it updates the package's billing dates.
Gather parameters
$cust_pkg is the package. $part_pkg is the package definition, either the “real” one for this package or a billing add-on. We temporarily set the cust_pkg->pkgpart property to match it so that we can call cust_pkg methods that delegate to “$self->part_pkg”. We then copy all the cust_pkg's fields into %hash.
$time is the “billing as of” date. %options is all the options that were passed to bill().
The other $param{} elements are accumulators passed from outside so that this method can add stuff to them.
$cust_location here isn't used for anything, because of tax engine refactoring.
The reference to freq_override is one last attempt to make sure term discounts aren't set up in a radically absurd way.
$lineitems is for remembering if we've done anything that requires creating a line item; if not then we'll exit without pushing one to the array.
@details will be used for any detail records that we attach to this line item. Those are additional lines of text shown below the line item on the invoice.
$cmp_time, as in bill(), is either “right now” or “the end of today” depending on the next-bill-ignore-time option.
sub _make_lines { my ($self, %params) = @_;
local($DEBUG) = $FS::cust_main::DEBUG if $FS::cust_main::DEBUG > $DEBUG;
my $part_pkg = $params{part_pkg} or die "no part_pkg specified"; my $cust_pkg = $params{cust_pkg} or die "no cust_pkg specified"; my $cust_location = $cust_pkg->tax_location; my $precommit_hooks = $params{precommit_hooks} or die "no precommit_hooks specified"; my $cust_bill_pkgs = $params{line_items} or die "no line buffer specified"; my $total_setup = $params{setup} or die "no setup accumulator specified"; my $total_recur = $params{recur} or die "no recur accumulator specified"; my $time = $params{'time'} or die "no time specified"; my (%options) = %{$params{options}};
my $tax_engine = $params{tax_engine};
if ( $part_pkg->freq ne '1' and ($options{'freq_override'} || 0) > 0 ) { # this should never happen die 'freq_override billing attempted on non-monthly package '. $cust_pkg->pkgnum; }
my $dbh = dbh; my $real_pkgpart = $params{real_pkgpart}; my %hash = $cust_pkg->hash; my $old_cust_pkg = new FS::cust_pkg \%hash;
my @details = (); my $lineitems = 0;
$cust_pkg->pkgpart($part_pkg->pkgpart);
my $cmp_time = ( $conf->exists('next-bill-ignore-time') ? day_end( $time ) : $time );
Charge setup fee and set setup date
Setup fees are charged the first time a package is billed. %setup_param will be passed to the calc_setup() method; because of how discounts work, it needs to know if this is an add-on package, and to report if it's applying any discounts. $setup and $unitsetup will become those fields on the cust_bill_pkg record.
The conditions for charging the setup fee are exactly as described in the inline comment. Note that a package that's canceled before first billing will never be charged a setup fee (or any other fee). We assume the package was ordered by mistake.
Note also that if the package has an expire date <= $cmp_time then it won't be charged. Normally the package would be canceled already. This check is for the case where the package isn't expired yet (in real time) but is being billed for a date after it will expire. For example, if a package has a next bill date of Feb 1, but an expiration date of Jan 31, and you're running freeside-daily -y 3 to print invoices 3 days in advance, then billing will trigger for the customer on Jan 28. The package won't be canceled yet. However, we should not preprint an invoice for this package because it will expire before its real billing date arrives. This is a real case that has come up.
Possibly we should check adjourn dates also, since a suspended package is not supposed to get set up.
### # bill setup ###
my $setup = 0; my $unitsetup = 0; my @setup_discounts = (); my %setup_param = ( 'discounts' => \@setup_discounts, 'real_pkgpart' => $params{real_pkgpart} ); my $setup_billed_currency = ; my $setup_billed_amount = 0; # Conditions for setting setup date and charging the setup fee: # - this is not a recurring-only billing run # - and the package is not currently being canceled # - and, unless we're specifically told otherwise via 'resetup': # - it doesn't already HAVE a setup date # - or a start date in the future # - and it's not suspended # - and it doesn't have an expire date in the past # # The "disable_setup_suspended" option is now obsolete; we never set the # setup date on a suspended package. if ( ! $options{recurring_only} and ! $options{cancel} and ( $options{'resetup'} || ( ! $cust_pkg->setup && ( ! $cust_pkg->start_date || $cust_pkg->start_date <= $cmp_time ) && ( ! $cust_pkg->getfield('susp') ) ) ) and ( ! $cust_pkg->expire || $cust_pkg->expire > $cmp_time ) ) {
warn " bill setup\n" if $DEBUG > 1;
If 'waive_setup' is set on the package then skip this step.
Calculate the setup fee. FS::cust_pkg->calc_setup just delegates to its part_pkg (which we've overridden if we're processing an add-on package right now). The following part_pkg classes have a calc_setup method:
- flat and recur_Common: Calls prorate_setup(), about which [#Deferred prorate see below]. Adds any “additional_info” strings to @details (this is used for putting notes on one-time charges). Calls base_setup() to get the setup fee from the package definition, then calc_discount() to apply any discounts, then multiplies the result by quantity. Almost all packages use this logic.
- currency_fixed: Calls prorate_setup(), then base_setup() (which does the currency conversion). Oddly, it doesn't multiply by quantity; this is probably a bug, since it does multiply by quantity in calc_recur().
- delayed_Mixin: If the 'delay_setup' option is off, pushes back the package bill date by some number of days so that the start of recurring billing is delayed. Then calls whatever calc_setup() method would otherwise apply.
- rt_field and (the obsolete) voip_sqlradacct: Returns the 'setup_fee' package option, without multiplying by quantity or applying discounts.
Also calculate the per-unit setup fee (for displaying on the invoice). The base_setup() method, by definition, returns the non-discounted per-unit setup fee.
Currency packages also return (via $setup_param) the currency and amount before doing conversion; we remember those here and will put them on the invoice later.
unless ( $cust_pkg->waive_setup ) { $lineitems++;
$setup = eval { $cust_pkg->calc_setup( $time, \@details, \%setup_param ) }; return "$@ running calc_setup for $cust_pkg\n" if $@;
# Only increment unitsetup here if there IS a setup fee. # prorate_defer_bill may cause calc_setup on a setup-stage package # to return zero, and the setup fee to be charged later. (This happens # when it's first billed on the prorate cutoff day. RT#31276.) if ( $setup ) { $unitsetup = $cust_pkg->base_setup() || $setup; #XXX uuh }
if ( $setup_param{'billed_currency'} ) { $setup_billed_currency = delete $setup_param{'billed_currency'}; $setup_billed_amount = delete $setup_param{'billed_amount'}; } }
Assign the setup date. The package may already have a setup date (because of 'resetup'). Otherwise, if it has a designated start date, use that (and clear the start date). Otherwise, it started billing now so set it to now.
if ( $cust_pkg->get('setup') ) { # don't change it } elsif ( $cust_pkg->get('start_date') ) { # this allows start_date to be used to set the first bill date $cust_pkg->set('setup', $cust_pkg->get('start_date')); } else { # if unspecified, start it right now $cust_pkg->set('setup', $time); }
$cust_pkg->setfield('start_date', ) if $cust_pkg->start_date;
}
Deferred prorate
This is an optional mode for prorate packages (and flat monthly packages using sync_bill_date), enabled by the prorate_defer_bill option. Rather than charging a fractional month at setup time, the package is billed on the first cutoff day for a full month plus the fractional month preceding the cutoff day.
In packages that are eligible for deferred prorate, calc_setup() calls the calc_prorate() method, which checks whether the flag is present. If so, calc_prorate sets the setup date (to now) and the next bill date (to the upcoming cutoff day) but doesn't set last_bill because the package hasn't really been billed. Then it returns a true value, which tells calc_setup not to charge a setup fee. The package will then not be charged a recurring fee either, because the next bill date is in the future.
(If the setup date is itself a cutoff day, then calc_prorate will ignore the flag and bill the package for a full month as usual. It is highly recommended that you use the prorate_round_day option in combination with this so that the “full month” really is a full month.)
When the first recurring billing date arrives, calc_recur() notices that there's a bill date but no last_bill, and charges for the partial month in the past and full month in the future. (The start and end dates on the line item will reflect this.) It will also call calc_setup() and apply any setup fee defined by the package.
Charge the recurring fee and adjust the bill date
The comment here explains the logic for whether to charge a recurring fee.
At this point, if it still has a start_date, then either the start_date is in the future or the package couldn't start billing for some other reason (like that it's on hold).
See the note above about the expire date; this part is slightly different in that if the package actually has expired and we're billing it on cancellation, then charging a recurring fee is allowed.
### # bill recurring fee ###
my $recur = 0; my $unitrecur = 0; my @recur_discounts = (); my $recur_billed_currency = ; my $recur_billed_amount = 0; my $sdate;
my $override_quantity;
# Conditions for billing the recurring fee: # - the package doesn't have a future start date # - and it's not suspended # - unless suspend_bill is enabled on the package or package def # - but still not, if the package is on hold # - or it's suspended for a delayed cancellation # - and its next bill date is in the past # - or it doesn't have a next bill date yet # - or it's a one-time charge # - or it's a CDR plan with the "bill_every_call" option # - or it's being canceled # - and it doesn't have an expire date in the past (this can happen with # advance billing) # - again, unless it's being canceled if ( ! $cust_pkg->start_date and ( ! $cust_pkg->susp || ( $cust_pkg->susp != $cust_pkg->order_date && ( $cust_pkg->option('suspend_bill',1) || ( $part_pkg->option('suspend_bill', 1) && ! $cust_pkg->option('no_suspend_bill',1) ) ) ) || $cust_pkg->is_status_delay_cancel ) and ( $part_pkg->freq ne '0' && ( $cust_pkg->bill || 0 ) <= $cmp_time ) || ( $part_pkg->plan eq 'voip_cdr' && $part_pkg->option('bill_every_call') ) || $options{cancel}
and ( ! $cust_pkg->expire || $cust_pkg->expire > $cmp_time || $options{cancel} ) ) {
Reset the package's usage cap if it has one:
- For voip_cdr, this is for “usage pools”. It clears all cdr_cust_pkg_usage records from the package and resets the cust_pkg_usage's minutes remaining.
- For flat (and other flat-like packages), this finds svc_accts attached to the package and resets their byte usage counters. This is for RADIUS services with data caps.
# XXX should this be a package event? probably. events are called # at collection time at the moment, though... $part_pkg->reset_usage($cust_pkg, 'debug'=>$DEBUG) if $part_pkg->can('reset_usage') && !$options{'no_usage_reset'}; #don't want to reset usage just cause we want a line item?? #&& $part_pkg->pkgpart == $real_pkgpart;
At this point we've decided to charge a recurring fee, so increment $lineitems.
“$sdate” is the effective billing date. It represents the end of the previous cycle and the start of the next. The next bill date will be set to $sdate + one billing period.
For a new package, $sdate is the setup date (so, the start date if there was one); if it's been billed before then it's the billing date.
Prorate math will normally charge for the period from $sdate to the next prorate cycle date after that. After that, it will set the next bill date to the prorate cycle date, so that future billing cyles run from one prorate cycle date to the next. The result is that the customer is charged for a partial month, then for full months after that.
$increment_next_bill is, yes, a flag saying to increment the bill date. We won't increment for one-time charges, or packages that are being canceled. The check for the bill date <= $cmp_time is now redundant, since we won't bill the package at all.
warn " bill recur\n" if $DEBUG > 1; $lineitems++;
# XXX shared with $recur_prog $sdate = ( $options{cancel} ? $cust_pkg->last_bill : $cust_pkg->bill ) || $cust_pkg->setup || $time;
#over two params! lets at least switch to a hashref for the rest... my $increment_next_bill = ( $part_pkg->freq ne '0' && ( $cust_pkg->getfield('bill') || 0 ) <= $cmp_time && !$options{cancel} );
The most important part of the entire module: calculate the recurring fee. Call FS::part_pkg::calc_recur (or calc_cancel, if we're canceling the package).
This is not a well-behaved interface. The arguments are all passed by reference (they're used to pass values back out), and they include the contents of %setup_param, so packages can use the %param hashref argument to pass notes between the calc_setup and calc_recur stages. Discounts, especially, use this.
- $sdate is the effective bill date, @details is the array of line item details, and %param contains everything else.
- 'discounts' is an arrayref of objects that will be inserted with the line item to track which discounts applied.
- 'precommit_hooks' and 'real_pkgpart' are used to keep track of the months remaining on term-limited discounts: if real_pkgpart matches the pkgpart of the package, then it's not an add-on package, so a month (or other amount) should be deducted. In that case a callback will be appended to precommit_hooks to do that at the end.
- 'freq_override' should be ignored.
- 'setup_fee' allows calc_recur to charge a setup fee. This seems bizarre but is needed for certain prorate cases.
- 'increment_next_bill' is checked in recur_Common to decide whether to apply the fixed recurring charge for voip_cdr and other usage-based packages. The intent seems to be to not bill those charges on cancellation.
my %param = ( %setup_param, 'precommit_hooks' => $precommit_hooks, 'increment_next_bill' => $increment_next_bill, 'discounts' => \@recur_discounts, 'real_pkgpart' => $real_pkgpart, 'freq_override' => $options{freq_override} || , 'setup_fee' => 0, );
my $method = $options{cancel} ? 'calc_cancel' : 'calc_recur';
# There may be some part_pkg for which this is wrong. Only those # which can_discount are supported. # (the UI should prevent adding discounts to these at the moment)
warn "calling $method on cust_pkg ". $cust_pkg->pkgnum. " for pkgpart ". $cust_pkg->pkgpart. " with params ". join(' / ', map "$_=>$param{$_}", keys %param). "\n" if $DEBUG > 2;
$recur = eval { $cust_pkg->$method( \$sdate, \@details, \%param ) }; return "$@ running $method for $cust_pkg\n" if ( $@ );
Handle a special case where calc_cancel() finds that the package isn't eligible to be billed on cancellation. This affects the decision about whether to create a line item at all.
if ($recur eq 'NOTHING') { # then calc_cancel (or calc_recur but that's not used) has declined to # generate a recurring lineitem at all. treat this as zero, but also # try not to generate a lineitem. $recur = 0; $lineitems--; }
Process more things returned by calc_recur. Ask the package for the unit recurring charge (passing $sdate because packages with an intro rate will report a different amount in the intro period).
If calc_recur did a currency conversion, record how it happened.
If calc_recur reported a quantity, record that quantity and adjust unitrecur. sql_external packages can do this.
#base_cancel??? $unitrecur = $cust_pkg->base_recur( \$sdate ) || $recur; #XXX uuh, better
if ( $param{'billed_currency'} ) { $recur_billed_currency = delete $param{'billed_currency'}; $recur_billed_amount = delete $param{'billed_amount'}; }
if ( $param{'override_quantity'} ) { $override_quantity = $param{'override_quantity'}; $unitrecur = $recur / $override_quantity; }
The other most important part of the module: increment the next bill date so that the package cycles.
As always in Freeside, adding some period to a bill date is done by the FS::part_pkg->add_freq method.
Deal with the case where this is a supplemental package. A supplemental package is “geared” to its main package: there's a fixed frequency ratio between them so that every N cycles they bill on the same day. If it's an integer multiple then that's easy (just add the main package period that man times). Otherwise, check whether the next bill is when they're due to sync up, and if so, sync them. This uses a rather crude “they're within 15 days” heuristic.
Otherwise, just call add_freq() using this package's frequency. Note that if the package is being prorated, $sdate will already have been moved to a prorate cycle date so that adding one month yields the right result.
if ( $increment_next_bill ) {
my $next_bill;
if ( my $main_pkg = $cust_pkg->main_pkg ) { # supplemental package # to keep in sync with the main package, simulate billing at # its frequency my $main_pkg_freq = $main_pkg->part_pkg->freq; my $supp_pkg_freq = $part_pkg->freq; if ( $supp_pkg_freq == 0 or $main_pkg_freq == 0 ) { # the UI should prevent setting up packages like this, but just # in case return "unable to calculate supplemental package period ratio"; } my $ratio = $supp_pkg_freq / $main_pkg_freq; if ( $ratio == int($ratio) ) { # simple case: main package is X months, supp package is X*A months, # advance supp package to where the main package will be in A cycles. $next_bill = $sdate; for (1..$ratio) { $next_bill = $part_pkg->add_freq( $next_bill, $main_pkg_freq ); } } else { # harder case: main package is X months, supp package is Y months. # advance supp package by Y months. then if they're within half a # month of each other, resync them. this may result in the period # not being exactly Y months. $next_bill = $part_pkg->add_freq( $sdate, $supp_pkg_freq ); my $main_next_bill = $main_pkg->bill; if ( $main_pkg->bill <= $time ) { # then the main package has not yet been billed on this cycle; # predict what its bill date will be. $main_next_bill = $part_pkg->add_freq( $main_next_bill, $main_pkg_freq ); } if ( abs($main_next_bill - $next_bill) < 86400*15 ) { $next_bill = $main_next_bill; } }
} else { # the normal case, not a supplemental package $next_bill = $part_pkg->add_freq($sdate, $options{freq_override} || 0); return "unparsable frequency: ". ($options{freq_override} || $part_pkg->freq) if $next_bill == -1; }
Set the “last bill” date to either the previous value of “next bill” or the setup date.
Also, if there's any second-pass setup fee, apply that to the line item.
'discount_left_setup' is a register used by the discount code for any flat-amount discount that's available for applying to the setup fee. Apply it here. This probably should set up a cust_bill_pkg_discount record but it's such an obscure case that we haven't needed it yet.
#pro-rating magic - if $recur_prog fiddled $sdate, want to use that # only for figuring next bill date, nothing else, so, reset $sdate again # here $sdate = $cust_pkg->bill || $cust_pkg->setup || $time; #no need, its in $hash{last_bill}# my $last_bill = $cust_pkg->last_bill; $cust_pkg->last_bill($sdate);
$cust_pkg->setfield('bill', $next_bill );
}
if ( $param{'setup_fee'} ) { # Add an additional setup fee at the billing stage. # Used for prorate_defer_bill. $setup += $param{'setup_fee'}; $unitsetup = $cust_pkg->base_setup(); $lineitems++; }
if ( defined $param{'discount_left_setup'} ) { foreach my $discount_setup ( values %{$param{'discount_left_setup'}} ) { $setup -= $discount_setup; } } } # end of recurring fee
warn "\$setup is undefined" unless defined($setup); warn "\$recur is undefined" unless defined($recur); warn "\$cust_pkg->bill is undefined" unless defined($cust_pkg->bill);
Construct a cust_bill_pkg record
If $lineitems is zero then billing was entirely skipped and there's nothing to do.
Otherwise, save the cust_pkg record with the new bill and last_bill dates. Only do that when processing the base package; add-on packages don't increment the bill date.
Clean up $setup and $recur. The 'allow_negative_charges' config is no longer used.
### # If there's line items, create em cust_bill_pkg records # If $cust_pkg has been modified, update it (if we're a real pkgpart) ###
if ( $lineitems ) {
if ( $cust_pkg->modified && $cust_pkg->pkgpart == $real_pkgpart ) { # hmm.. and if just the options are modified in some weird price plan?
warn " package ". $cust_pkg->pkgnum. " modified; updating\n" if $DEBUG >1;
my $error = $cust_pkg->replace( $old_cust_pkg, 'depend_jobnum'=>$options{depend_jobnum}, 'options' => { $cust_pkg->options }, ) unless $options{no_commit}; return "Error modifying pkgnum ". $cust_pkg->pkgnum. ": $error" if $error; #just in case }
$setup = sprintf( "%.2f", $setup ); $recur = sprintf( "%.2f", $recur ); if ( $setup < 0 && ! $conf->exists('allow_negative_charges') ) { return "negative setup $setup for pkgnum ". $cust_pkg->pkgnum; } if ( $recur < 0 && ! $conf->exists('allow_negative_charges') ) { return "negative recur $recur for pkgnum ". $cust_pkg->pkgnum; }
One more check. If the setup and recurring fees are both zero, we might skip creating a line item. It depends. Create a zero-dollar line item if:
- The setup or recur is zero because it was discounted to zero, and the discount-show-always config is on. The invoice will then show the applied discount.
- This is the main package in a bundle. Its add-on packages will add hidden line items for the charges, but we need this one so that the bundle is shown with the correct package name.
- Either the cust_pkg or the part_pkg has the 'setup_show_zero' or 'recur_show_zero' flag. (It's not clear to me why these are separate flags, since they have exactly the same effect.)
my $discount_show_always = $conf->exists('discount-show-always') && ( ($setup == 0 && scalar(@setup_discounts)) || ($recur == 0 && scalar(@recur_discounts)) );
if ( $setup != 0 || $recur != 0 || (!$part_pkg->hidden && $options{has_hidden}) #include some $0 lines || $discount_show_always || ($setup == 0 && $cust_pkg->_X_show_zero('setup')) || ($recur == 0 && $cust_pkg->_X_show_zero('recur')) ) {
warn " charges (setup=$setup, recur=$recur); adding line items\n" if $DEBUG > 1;
Find any package details that are set to display as “invoice details” and add them to the list.
Then actually create the cust_bill_pkg. Huzzah!
my @cust_pkg_detail = map { $_->detail } $cust_pkg->cust_pkg_detail('I'); if ( $DEBUG > 1 ) { warn " adding customer package invoice detail: $_\n" foreach @cust_pkg_detail; } push @details, @cust_pkg_detail; my $cust_bill_pkg = new FS::cust_bill_pkg { 'pkgnum' => $cust_pkg->pkgnum, 'setup' => $setup, 'unitsetup' => sprintf('%.2f', $unitsetup), 'setup_billed_currency' => $setup_billed_currency, 'setup_billed_amount' => $setup_billed_amount, 'recur' => $recur, 'unitrecur' => sprintf('%.2f', $unitrecur), 'recur_billed_currency' => $recur_billed_currency, 'recur_billed_amount' => $recur_billed_amount, 'quantity' => $override_quantity || $cust_pkg->quantity, 'details' => \@details, 'discounts' => [ @setup_discounts, @recur_discounts ], 'hidden' => $part_pkg->hidden, 'freq' => $part_pkg->freq, };
Determine the start and end of the billing period. Usually it's “last bill to current bill”, or “current bill to next bill”, but there are a couple of special cases. Also record which part_pkg this line item was generated for, if it's an add-on package.
if ( $part_pkg->option('prorate_defer_bill',1) and !$hash{last_bill} ) { # both preceding and upcoming, technically $cust_bill_pkg->sdate( $cust_pkg->setup ); $cust_bill_pkg->edate( $cust_pkg->bill ); } elsif ( $part_pkg->recur_temporality eq 'preceding' ) { $cust_bill_pkg->sdate( $hash{last_bill} ); $cust_bill_pkg->edate( $sdate - 86399 ); #60s*60m*24h-1 $cust_bill_pkg->edate( $time ) if $options{cancel}; } else { #if ( $part_pkg->recur_temporality eq 'upcoming' ) $cust_bill_pkg->sdate( $sdate ); $cust_bill_pkg->edate( $cust_pkg->bill ); #$cust_bill_pkg->edate( $time ) if $options{cancel}; }
$cust_bill_pkg->pkgpart_override($part_pkg->pkgpart) unless $part_pkg->pkgpart == $real_pkgpart;
Record the results in the right places: add setup and recur to totals, give the cust_bill_pkg to the tax calculator, and add it to the line item array.
$$total_setup += $setup; $$total_recur += $recur;
### # handle taxes ###
my $error = $tax_engine->add_sale($cust_bill_pkg); return $error if $error;
$cust_bill_pkg->set_display( part_pkg => $part_pkg, real_pkgpart => $real_pkgpart, );
push @$cust_bill_pkgs, $cust_bill_pkg;
} #if $setup != 0 || $recur != 0
} #if $line_items
;
}
And we're done!