Navigation

2/26/2015

OPAC SQL: speed of queries

Here is a way to speed up SQL queries in Innovative Interfaces:




Here’s a version of the query that completes in about 1500 ms on our database, or about 0.017 ms per record, which, if I’m right, should shave an additional couple of seconds off for you (7650 ms):

SELECT
  rm.record_type_code || rm.record_num AS record_num,
  email.field_content AS email,
  pname.last_name || ', ' || pname.first_name || ' ' || pname.middle_name AS name,
  pr.home_library_code
FROM
  sierra_view.record_metadata rm
    JOIN sierra_view.patron_record pr
      ON rm.record_type_code = 'p' AND rm.id = pr.record_id
    JOIN sierra_view.varfield email
      ON email.varfield_type_code = 'z' AND email.record_id = pr.record_id
    JOIN sierra_view.patron_record_fullname pname
      ON pname.patron_record_id = pr.record_id
WHERE
  email.field_content !~ '[^@]+@[^@]+[.][^@]+';

The main difference here between this one and yours and Joe’s is that it doesn’t use any of the *_view tables. Those are constructed views that III created for convenience, but in my experience they’re almost always slower than just picking whichever tables you need and joining manually. Of course, 7-8 seconds is still a very long response time for a UI, so doing something like what Joshua suggested may still be necessary.

As an aside, if you’re not familiar with using EXPLAIN and EXPLAIN ANALYZE, these are very helpful tools for figuring out what’s going on in a particular query. Their output is a bit cryptic, but there are several helpful tutorials/explanations available. I always have to refer back to the Postgresql documentation, here:


If you’re using pg-admin, it can give you a nice visual representation (in the Query window, go to Query / Explain and Query / Explain Analyze) that’s a bit easier to parse than the raw output.

Good luck!

Jason Thomale
User Interfaces Unit, UNT Libraries


From: Boggs, John Sent: Wednesday, February 25, 2015 12:07 PM
To: Sierra Users List
Subject: [sierra] Speed up an SQL query?

Hi all,

I'm trying to put together a canned report for my circ folks of badly-formed email addresses. The report takes about 15 seconds to run, which isn't really all that long but it does lead folks waiting for a web page to click on the link again thinking maybe their click didn't work -- and that just causes the query to start all over again. I have a much more complex query, over much more data, for items in transit too long that runs in about 3 seconds and I'm puzzled as to why the patron one would run so much slower.  We have 2.5 million items and only 450,000 patrons.

I'm attaching the query in case anybody sees anything I'm missing. I have tried selecting from each of the three tables that I'm using and joining in a different orders, but it all takes at least 15 seconds. 

Thanks for any suggestions!

SELECT
  'p' || pat_view.record_num || 'a',
  email.field_content,
  name.last_name || ',' || name.first_name || ' ' || name.middle_name,
  pat_view.home_library_code
FROM
  sierra_view.patron_view AS pat_view
JOIN sierra_view.varfield_view AS email
  ON ( pat_view.record_num = email.record_num AND
       email.record_type_code = 'p' AND
       email.varfield_type_code = 'z' )

JOIN sierra_view.patron_record_fullname AS name
  ON (pat_view.id = name.patron_record_id)

WHERE
  email.field_content !~ '[^@]+@[^@]+[.][^@]+';

2/12/2015

Apple target disk mode

Have Apple laptop that is not booting, want to connect to working Mac that has thunderbolt connection:


If you have two Mac computers with FireWire or Thunderbolt ports, you can connect them so that one of them appears as an external hard disk on the other. This is called “target disk mode.”
  1. Connect the two computers with a FireWire or Thunderbolt cable.
  2. Start up the computer to be used as a disk in target disk mode:
    • If the computer is off, start it up while holding down the T key.
    • If the computer is on, choose Apple menu > System Preferences, click Startup Disk, and then click Target Disk Mode.
      When the computer has started up, a disk icon appears on the desktop of the other computer.
  3. Transfer files by dragging them to and from the disk.
  4. Eject the disk by dragging its icon to the Trash. While you drag, the Trash icon changes to an Eject icon.
  5. On the computer you used as a disk, push the power button to shut it down, and then disconnect the cable.


Was able to save files from the laptop hard drive

http://support.apple.com/kb/PH10725


2/09/2015

SQL to get patrons who owe more than $25 in fines

From Sierra Listserv, code to output library patrons who owe a certain amount in fines:




ADDING PATRON NAMES TO OUTPUT

First, let's add patron names to your existing query. You start by adding a JOIN to sierra_view.patron_record_fullname. You than add a name column to your output. The trick here is that Sierra stores names as first_name, middle_name, and last_name in three separate columns. In order to output the full name, you need to concatenate the pieces together.

In the following query, you'll find the concatenated name in LINE 4. The additional JOIN is in LINES 21-24.
select
p.home_library_code,
p.record_num,
pn.last_name||', '||pn.first_name||' '||pn.middle_name,
p.barcode,
f.assessed_gmt,
f.paid_gmt,
f.charge_code,
f.item_charge_amt,
f.billing_fee_amt,
f.processing_fee_amt,
f.paid_amt,
f.title,
f.description
from sierra_view.fine as f
JOIN -- to patron view
sierra_view.patron_view AS p
ON
p.id = f.patron_record_id
and assessed_gmt <= '2014-03-31 00:00:00'
JOIN -- to patron name
sierra_view.patron_record_fullname AS pn
ON
pn.patron_record_id = p.id
;

ADDING A NESTED QUERY

In order to limit your output to only those patrons who have fines and fees that were assessed on or before 2013-03-31 and which total $25 or more, I think you'll need to do a nested query. The following query will give you a list of patron record ids that match the desired criteria -
SELECT
  patron_record_id
FROM
  sierra_view.fine
WHERE
  DATE(assessed_gmt) <= '2014-03-31'
GROUP BY
  1
HAVING
  SUM(item_charge_amt+processing_fee_amt+billing_fee_amt-paid_amt) >= '25'
;

You then nest that query into the larger query. Output from the larger query will be limited to only patrons whose patron record IDs are found in the nested query's result set. Note the WHERE clause (LINES 25-37) -
select
p.home_library_code,
p.record_num,
pn.last_name||', '||pn.first_name||' '||pn.middle_name,
p.barcode,
f.assessed_gmt,
f.paid_gmt,
f.charge_code,
f.item_charge_amt,
f.billing_fee_amt,
f.processing_fee_amt,
f.paid_amt,
f.title,
f.description
from sierra_view.fine as f
JOIN -- to patron view
sierra_view.patron_view AS p
ON
p.id = f.patron_record_id
and assessed_gmt <= '2014-03-31 00:00:00'
JOIN -- to patron name
sierra_view.patron_record_fullname AS pn
ON
pn.patron_record_id = p.id
WHERE
f.patron_record_id in
(
  SELECT
  patron_record_id
  FROM
  sierra_view.fine
  WHERE
  DATE(assessed_gmt) <= '2014-03-31'
  GROUP BY 1
  HAVING
  SUM(item_charge_amt+processing_fee_amt+billing_fee_amt-paid_amt) >= '25'
)
;

OUTPUT AND CODE FORMAT

I would tend to want to format my output for dates and currency and I would want to add textual descriptions for charge_code and home_library_code. I would also tidy up my query code in some way that makes it easier for me to read later. Here are my suggestions. Make use of whatever works for you.
  1. I replaced the bare home library code with a branch name. I got this by adding another JOIN to sierra_view.location_myuser
  2. I used the DATE() function on f.assessed_gmt to quickly strip off the transaction time and timezone, leaving only the date
  3. I added a CASE statement to provide text for charge_code. There is no view that holds these descriptors (as far as I can tell). I got them from Sierra DNA and have to hope that they don't change over time.
  4. I formatted the currency fields to use only two decimal places and to show the dollar sign
  5. I added a "Balance Owing" column wherein I add the three charge columns and subtract the amount already paid
-- ===============================================================================
-- Patrons with $25 or more in fines assessed on or before 2014-03-31
--    Detail of fines owing
-- Brent Searle. Langara College. 2015-02-06
-- ===============================================================================
SELECT
  ln.name                                 AS "Home Library",
  p.record_num                            AS "Patron Number",
  TRIM(pn.last_name||', '||
  pn.first_name||' '||
  pn.middle_name)                         AS "Patron Name",
  p.barcode                               AS "Patron Barcode",
  DATE(f.assessed_gmt)                    AS "Date Assessed",
  CASE
    WHEN f.charge_code = '1' THEN 'manual charge'
    WHEN f.charge_code = '2' THEN 'overdue'
    WHEN f.charge_code = '3' THEN 'replacement'
    WHEN f.charge_code = '4' THEN 'adjustment'
    WHEN f.charge_code = '5' THEN 'lost item'
    WHEN f.charge_code = '6' THEN 'overdue renewed'
    WHEN f.charge_code = '7' THEN 'rental'
    WHEN f.charge_code = '8' THEN 'rental adjustment'
    WHEN f.charge_code = '9' THEN 'debit'
    WHEN f.charge_code = 'a' THEN 'notice'
    WHEN f.charge_code = 'b' THEN 'credit card'
    WHEN f.charge_code = 'p' THEN 'program'
    ELSE 'unexpected charge code - '||f.charge_code
  END                                     AS "Charge Type",
  TO_CHAR(f.item_charge_amt,'$990.99')    AS "Amount Charged",
  TO_CHAR(f.billing_fee_amt,'$990.99')    AS "Billing Fee",
  TO_CHAR(f.processing_fee_amt,'$990.99') AS "Processing Fee",
  TO_CHAR(f.paid_amt,'$990.99')           AS "Amount Paid",
  TO_CHAR(f.item_charge_amt +
  f.billing_fee_amt +
  f.processing_fee_amt -
  f.paid_amt,'$990.99')                   AS "Balance Owing",
  f.title                                 AS "Title",
  f.description                           AS "Fine Description"
FROM
  sierra_view.fine                        as f
JOIN -- to patron view
  sierra_view.patron_view                 AS p
  ON
  p.id = f.patron_record_id
JOIN
  sierra_view.patron_record_fullname      AS pn
  ON
  pn.patron_record_id = p.id
JOIN
  sierra_view.location_myuser             AS ln
  ON
  ln.code = p.home_library_code
WHERE
  DATE(f.assessed_gmt) <= '2014-03-31'    -- specific date may need editing
  AND
  p.owed_amt >= '25'
  AND
  f.patron_record_id IN 
  (
    SELECT
      patron_record_id
    FROM
      sierra_view.fine
    WHERE
      DATE(assessed_gmt) <= '2014-03-31'  -- specific date may need editing
    GROUP BY
      1
    HAVING
      SUM(item_charge_amt+processing_fee_amt+billing_fee_amt-paid_amt) >= '25'
  )
ORDER BY
  1,2,4
; 

You should be able to export the output to MS Excel and, by way of the SUBTOTAL function, subtotal on each change of patron number.


--
Brent Searle
Library Systems Manager
Langara College
Vancouver BC

On 2015-02-03 1:48 PM, Leach, Barbara wrote:
I’m hoping maybe one of you SQL gurus can help me with this SQL for Collection Agency candidates.  The following query lists fines assessed prior to a certain date (I used 3/31/14 to keep the results list small.  We went live on Sierra on 1/23/14)

Now, what would be really helpful is if it would sum all of the fines, and only include in the output patrons who owed $25 or more prior to the specified date.  This is way beyond my sql abilities. 

Also, I’d like to include the patron name, but I didn’t know how to add a second join to the patron name table. (I tried, but it didn’t work. )

select
p.home_library_code,
p.record_num,
p.barcode,
f.assessed_gmt,
f.paid_gmt,
f.charge_code,
f.item_charge_amt,
f.billing_fee_amt,
f.processing_fee_amt,
f.paid_amt,
f.title,
f.description
from sierra_view.fine as f
JOIN -- to patron view
sierra_view.patron_view AS p
ON
p.id = f.patron_record_id
and assessed_gmt <= '2014-03-31 00:00:00'
;

Thanks so much for your help!  We just can’t figure out a way to get Sierra to work to send patrons with fines to Collection Agency, so it doesn’t include current fines.  Any way we imagine involves a lot of manual staff work, that we’re just not used to having to do (our previous ILS did it for us perfectly)

III OPAC

2/07/2015

SQL to get past checkouts from Innovative ILS

We have turned off the last patron to checkout an item field, but we wanted to track down the person who borrowed a laptop charger (missing a part).  This info is stored in CIRC_TRANS table, generally 14 days worth, to see all: 





SELECT
  transaction_gmt
FROM
  sierra_view.circ_trans
ORDER BY
  1
LIMIT
  100
;



To locate one item by barcode, use this code:


SELECT
  circ_trans.transaction_gmt,
  circ_trans.patron_record_id,
  circ_trans.item_record_id,
  circ_trans.due_date_gmt,
  item_view.barcode,
  item_view.itype_code_num,
  patron_record_fullname.first_name,
  patron_record_fullname.last_name
FROM
  sierra_view.circ_trans,
  sierra_view.item_view,
  sierra_view.patron_view,
  sierra_view.patron_record_fullname
WHERE
  item_view.id = circ_trans.item_record_id AND
  patron_view.id = circ_trans.patron_record_id AND
  patron_record_fullname.patron_record_id = patron_view.id AND
  item_view.barcode = '11111103690302'

ORDER BY
  circ_trans.due_date_gmt ASC;


Barcode is CHAR field, so can use ::text to set the number to text, double quotes do not work  (PGadmin client)


To get daily stats on checkouts:  (via Andrew Hollingsworth)



Select Distinct
  count(*),
  Case sierra_view.circ_trans.op_code When 'o' Then 'Checkout'
    When 'n' Then 'Hold' When 'nb' Then 'Title Hold' When 'ni' Then 'Item Hold'
    When 'f' Then 'Hold Fufilled' When 'i' Then 'Checkin'
    When 'r' Then 'Renewal' When 'h' Then 'Recall Hold'
    When 'u' Then 'Use Count' When 'b' Then 'Booking'
    When 'hb' Then 'Hold Recall Bib'
    When 'hi' Then 'Hold Recall Item'
    When 'hv' Then 'Hold Recall Volume' End As transtype From
  sierra_view.circ_trans
  Where
  sierra_view.circ_trans.transaction_gmt Between '02/10/2015 00:00:00'
  And '02/11/2015 24:00:00'
Group By
  sierra_view.circ_trans.op_code
Order by
transtype asc