php - Memory usage increasing even after executing the process again


I have a symfony 1.4 task that opens a csv file with more than 200000 lines using splFileObject.

This is the code responsible for opening the file :

//open the CSV file for read
      $this->file = new SplFileObject($this->filePath, 'r');

      $this->limit = $this->options['limit'];

      /*       * read the file* */
      $i = $this->limit + 1;
      foreach ($this->file as $row)
        if ($this->file->valid())
          //initialize invoices and clients object

          //clear the row

          $this->key = $this->file->key();

          //add msisdn to msisdn array, and client to clients array
          self::$InvoicesArray[$this->msisdn] = $this->client;

          //get valid users from table, and save to payment_notifications

          if ($i % $this->options['count'] === 0)
            //launch process again with new $key
            $this->logSection('Payment Notification', $i. 'Lines Processed, Memory Usage : ' . memory_get_usage());
            $this->log($this->fileSys->execute("./symfony mediapi:send-payment-notifications --application=" . $this->options['application']
                    . " --env=" . $this->options['env'] .
                    " --limit=" . $i .
                    ' --count=' . $this->options['count']));



As you can see, I execute the process once limit reaches 1000 lines. the limit option is used to determine from which line in the file the current process must start.

$this->prepareAndSaveValidClients(); is a function that validates data with a select, prepares objects and saves data to the database... this is the prepareAndSaveValidClients() :

  public function prepareAndSaveValidClients() {

    //get valid clients from database
    $this->query = Doctrine_Query::create()
        ->select('p.gender, p.email2, u.username, u.first_name, u.last_name, u.email_address, u.is_active, p.msisdn, p.user_id, p.city_id, p.street, p.zipcode, p.msisdn_status')
        ->from('sfGuardUser u')
        ->innerJoin('u.Profile as p ON = p.user_id')
        ->whereIn('p.msisdn', $this->msisdn)
        ->whereIn('p.status', self::$AllowedStatus);

    //fetch valid clients result
    $this->results = $this->query->fetchArray();

    //unset the query to free memory

    //instanciat an object collection for payment_notifications
    $collection = new Doctrine_Collection("payment_notifications");

    //save valid clients in payment_notifications table
    if (!empty($this->results))
      foreach ($this->results as $key => $client)
        $invoice = self::$InvoicesArray[$client['Profile']['msisdn']];

        //prepare invoices

        //prepare userprofile

        $paymentNotifications = new paymentNotifications();


        //clear $client and $paymentNotifications

      //save the collection

      //freeing the collection

      //clear memory
      $this->results = array();

      self::$MsisdnArray = array();
      self::$InvoicesArray = array();


The problem is that Memory_Usage increases through different processes:

Process 1 : 1000 Lines Processed, Memory Usage : 40947336
Process 2 : 2000Lines Processed, Memory Usage : 69401208
Process 3 : 3000Lines Processed, Memory Usage : 98444272
Process 4: 4000Lines Processed, Memory Usage : 126308156

How can I start new process, and free the previous process memory usage ???

Thank you

People are also looking for solutions to the problem: regex - how can i preg_match a xml declaration using php?


Didn't find the answer?

Our community is visited by hundreds of web development professionals every day. Ask your question and get a quick answer for free.

Ask a Question

Write quick answer

Do you know the answer to this question? Write a quick response to it. With your help, we will make our community stronger.

Similar questions

Find the answer in similar questions on our website.