We have our Web server running on a MAC OS X machine while our Web reporting software, NetTracker, runs on a Windows 2003 server. We needed to get the data across everyday so our reporting could be real time minus one day. I ended up writing a shell script to move the data: Hopefully this will help out someone out there.
*NOTE: This assumes you are doing a daily dump of your Apache log files
cd /var/log/httpd
# On Sunday - Apache does a dump of the weeks worth of data
# we need to archive it
mv access_log.0.gz week_$DATETIME.gz
# Else rename the daily log file first -
mv access_log.*.gz log_$DATETIME.gz
# Now lets gunzip it
gunzip log_$DATETIME.gz
# Time to chmod for so right permissions come over to Windows
chmod 777 log_$DATETIME
#Now put it on your Windows Share
smbclient //YourServerName/Logs -UMyUserName%MyDomainName -W DomainName -c "prompt; mput log_$DATETIME; exit;"
exit 0
I still need to write some type of logging into the script incase there is an error. Also, there is a way to specify a encrypted file name for the password instead of having the password in your script, but I ended up changing the permission on the actual script file so only root could read it, root is the user running the cron job, so that should be enough security.
I love podcast, I listen to them on the commute to and from work. But, good podcast are hard to find, really hard to find. Below is my list of the top ten podcast. Updated on 2-22-06.
Hanselminutes - Scott Hanselman a ASP.NET Guru dives weekly into a range of technology. Very informative, quick to the point and entertaining.
TWIT - A weekly podcast about the week in technology held in a roundtable type discussion. It can get out of hand and off topic once in a while but still a weekly listen.
Web Developers Radio - Bi monthly podcast that focuses primarliy on the LAMP stack technology (Linux, Apache, mySQL, PHP, Perl). Also some good stuff on AJAX and Javascript.
IT Conversations - Not a weekly podcast I listen to. In fact, this is a podcast I go grab something from their archives when I run out of current podcast to listen to. Neverless, they have some really good stuff from previous popTech sessions.
Inside the Net - Amber Mac and Leo from Twit interview tech leaders.
Venture Voice - Greg Galant podcast that explores how entrepreneurs build their businesses and live their lives. His voice is a bit geeky but his the substance makes up for it.
.NET Rocks - Put on by franklins.net. A good podcast usually about .net technology.
As you can see I am 3 short of my top ten. Frankly, I haven’t found any worth putting on the list. If you know of good one, shoot me an email - tbolandatgmaildotcom
To some folks rss is this hot new technology that is changing our lives in technology. The truth is rss is a specific formated text file that people finally have standarized on. It’s the standarization that changes people lives, not neccessary the technology. Anyway, a while back I wanted to rss feed our knowledgebase. The knowledgebase articles are stored in a mysql database. I played around with using PEAR::PACKAGE::XML to do the job but found it to be a bit overkill. So I wrote my own xml builder - source code is below or you can download the php rss feed mysql database code. What is nice, is that you can RSS feed any database.
// prepare HTML text for use as UTF-8 character data in XML
function cleanText($intext) {
return utf8_encode(
htmlspecialchars(
stripslashes($intext)));
}
// set the file's content type and character set
// this must be called before any output
header("Content-Type: text/xml;charset=utf-8");
// retrieve database records
$db = mysql_connect("YourMySQLSERVER", "YOURUSERNAME", "YOURPASSWORD");
if (!$db)
{
error_log("Error: Could not connect to database in rss.php.");
exit;
}
// store items from the database in the $result1 array
mysql_select_db("knowledgebase");
$query1 = "SELECT faqarticles.question, faqarticles.content, SUBSTRING(content,1,600) as mycontent,
faqarticles.cat, faqarticles.id, faqarticles.date FROM faqarticles WHERE
faqarticles.approved != '1'
ORDER BY faqarticles.date DESC";
$result1 = mysql_query($query1);
$phpversion = phpversion();
// display RSS 2.0 channel information
ECHO <<
http://www.yourlinkgoeshere.com
Describe your RSS Feed Heredescription> en-us http://backend.userland.com/rss PHP/$phpversion
http://www.yoursite.com/images/YourLogo.gif
http://www.yoursite.com/index.php
140 60 Describe Your Image
END;
// loop through the array pulling database fields for each item
for ($i = 0; $i < mysql_num_rows($result1); $i++) {
@$row = mysql_fetch_array($result1);
$title = cleanText($row["question"]);
$link = "http://www.YourSite.com/kb/index.php?article=".$row["id"];
$description = $row["mycontent"];
//Replace Ugly HTML that got into the Knowledgebase with nothing
$desc_replace = array("
We have been doing a lot of asp.net 2.0 development lately. Here is how we are setting up the web.config file to hold our connection settings. First, set your web.config file Connection String settings as follows:
In your data access layer classes set the sqlconnection object to the web.config connection string thru the constructor. For VB.NET that is the class’s New() function. In C# it is the function named the same as the class. Below are both examples or download the C# version or VB.NET version directly.
Dim conn As New SqlConnection Dim sql As String Dim objSql As SqlCommand Dim MyDataReader As SqlDataReader
Public Sub New() If ConfigurationManager.ConnectionStrings("YourConnectionString") Is Nothing OrElse _ ConfigurationManager.ConnectionStrings("YourConnectionString").ConnectionString.Trim() = "" Then Throw New Exception("Connection Error") Else conn.ConnectionString = ConfigurationManager.ConnectionStrings("YourConnectionString").ConnectionString End If
End Sub
Public Function GetMSRP(ByVal productid As Int32) As String
Dim strUnitPrice As String = Nothing sql = "SELECT UnitPrice FROM Products WHERE productid = @productid" ‘Set SQL OBJECT objSql = New SqlCommand(sql, conn) ‘Add Parameters objSql.Parameters.AddWithValue("@productid", productid)
Try ‘Open Connection conn.Open() ‘Execute DataReader MyDataReader = objSql.ExecuteReader ‘Store Values in String Variables While (MyDataReader.Read()) strUnitPrice = MyDataReader.Item("UnitPrice") End While ‘Close Connection conn.Close() Catch ex As Exception ‘TODO HANDLE EX End Try Return strUnitPrice End Function
End Class
################### c# version
using Microsoft.VisualBasic; using System.Data; using System.Data.SqlClient; using System.Web.UI; public class DataAccess {
private SqlConnection conn = new SqlConnection();
private string sql;
private SqlCommand objSql;
private SqlDataReader MyDataReader;
public DataAccess() { if (((ConfigurationManager.ConnectionStrings("YourConnectionString") == null) || _)) { ConfigurationManager.ConnectionStrings("YourConnectionString").ConnectionString.Trim() = ""; throw new Exception("Connection Error"); } else { conn.ConnectionString = ConfigurationManager.ConnectionStrings("YourConnectionString").ConnectionString; } }
public string GetMSRP(Int32 productid) { string strUnitPrice = null; sql = "SELECT UnitPrice FROM Products WHERE productid = @productid"; objSql = new SqlCommand(sql, conn); // Add Parameters objSql.Parameters.AddWithValue("@productid", productid); try { // Open Connection conn.Open(); // Execute DataReader MyDataReader = objSql.ExecuteReader; // Store Values in String Variables while (MyDataReader.Read()) { strUnitPrice = MyDataReader.Item["UnitPrice"]; } // Close Connection conn.Close(); } catch (Exception ex) { // TODO HANDLE EX } return strUnitPrice; } }
After writing some functions or classes its good idea to make sure your code is executing in a timely fashion, especially for web applications. For PHP I use a function derived from the chronmeter class to look for bottle necks (the function is below or download it here).
We recently ran into a problem of a web form running particular slow. It had the following functionality.
Insert some data into 2 mysql tables
Grab some data from a Microsoft SQL Database
Read in a text file, replace some verbiage, then send out a couple of emails.
I architected the form and was letting one of our interns implement it. Everything seemed pretty straight forward - this code should fly. After the site went live, I checked out the form and found it running slow. Using the chronometer function I paired it down to our phpmailer. Turns out we were using php’s built in mail function and not phpmailer.
After switching to phpmailer and setting the option to smtp, I cut the runtime in half. I then realized we were sending out 2 emails and each time opening/closing an smtp connection. I refactored so only 1 connection was opened. Here are the results of the average form runtime:
PHP’s built in mail function: Avg: 3 seconds.
PHPMAILER function with 2 connections: 1.5 seconds.
PHPMAILER function with 1 connection: .105 seconds
To use the chronometer function, simply call it before a class or function call and then call it right after it .
if ($CHRONO_STARTTIME > 0) {
/* Stop the chronometer : return the amount of time since it was started,
in ms with a precision of 3 decimal places, and reset the start time.
We could factor the multiplication by 1000 (which converts seconds
into milliseconds) to save memory, but considering that floats can
reach e+308 but only carry 14 decimals, this is certainly more precise */
I am trying to get better at keeping track of my time. Where I use to work - Bridge WorldWide - I had to bill all of my time to specific job numbers or clients. It was one of the worst thing about the job (I guess if that is the worst thing about Bridge, it is a great place to work). A lot of the programmers there kept written time sheets. That was way beyond techy for me so for a while I used Outlook’s calendar. Now I use I use a free .NET program named Time Snapper. It sits in your system tray and takes a snap shot of your desktop every x seconds. Whats even better, it has a browser that lets you ”watch” your week. Don’t worry about idle time, the program doesn’t take a snap if your system has been idle so many x seconds. I love it. I set mine to take a snap every 4 minutes and produce a JPG. Typically a day’s worth of data is around 10 MB. Time Snapper has an option to clean up itself after so many days. Check it out.
