Website Offline Copy

Ever wanted to have an entire website available as a local copy? I did yesterday.

There's a website with a forum that is about to be closed and that contains lots of useful info. So I thought: "Why not save it?"

I gave some tools a try but I could not get them to get all the files. Then I decided to use the good old wget.

I logged in on an old linux machine and after reading the man page for a while I used the following command to download the site:


$ wget \
     --recursive \
     --no-clobber \
     --page-requisites \
     --html-extension \
     --convert-links \
     --restrict-file-names=windows \
     --domains example.com \
     --no-parent \
         www.example.com






In case the part of the site is protected with a log on page you may use cookies to access the content.

Firstly get the cookie with the following command: 






$ wget \


          --save-cookies cookies.txt \


          --post-data 'user=yourusername&password=yourpassword' \


          http://authurl










You can get the names of the user and password field variables that you need to post and the authentication URL by examining the logon form.


Then you can add --load-cookies cookies.txt and --keep-session-cookies parameters to use the cookies.
























































Popular posts from this blog









Domain Controller Machine Password Reset




















Image







On my lab environment, I've configured two Active Directory sites since most enterprises have offices in more that one places. My lab however is not running 24/7 and the domain controllers in the second site are rarely turned on in order to save resources.   This leads to issues with the Active Directory replication such as the "The target principal name is incorrect" error when I execute:  repadmin /syncall /AdeP. To remedy the issue, we have to reset the machine password of the domain controller that has been offline.      First off, we are going to stop and disable the Kerberos Key Distribution Center (kdc) service on the problematic domain controller, in our case DC4.   There may be some tickets in the cache so we should also clear them using klist purge      Now it's time to change the machine password of the domain controller using the command   netdom resetpwd /s:dc3 /ud:lab\administrator /pd:*   Replace the "lab\administrator" with an account on your...








Keep reading










Managing Active Directory User Certificates using PowerShell





















I first came across user certificates when I was working with email certificates a few years ago and I have to admit that I had trouble updating the certificates on the objects!   Most organizations have a Microsoft Active Directory Certification Authority that issues the certificates used internally. When a certificate is issued to a user, the Microsoft Certificate Service saves the public key in Active Directory. The userCertificate attribute is a multi-valued attribute that contains the DER-encoded X509v3 certificates issued to the user.   Although we rarely need to pay attention to this attribute, there are cases where we have to update it. To make things easier, I've written PowerShell functions to Get, Remove, Import and Export the certificates on that field.   To get the list of certificates for an object, use the Get-ActiveDirectoryObjectCertificate function:    PS C:\> Get-ActiveDirectoryObjectCertificate -UserPrincipalName cpolydorou@lab.local   DistinguishedName  ...








Keep reading










Configuring a Certificate on Exchange Receive Connector





















 Today's article is about configuring Exchange receive connectors with specific certificates.   Out of the box, Exchange uses self signed certificates to provide TLS secured mail flow. This will definitely be an issue if you expose the SMTP protocol to client computers since they won't trust the certificate.   In this article we are going to configure a certificate that was issued by a third part authority to the Client Frontend receive connector.   We'll start with getting the thumbprint of the certificate using the Get-ExchangeCertificate cmdlet:        [PS]    C:\> Get-ExchangeCertificate   Thumbprint                               Subject  ----------                               -------  241B864DC82C664FECBA18B8D54987AAFB65B4C2  CN=*.lab.com, ...  D4D210886B34E690191A1F008C78FDD0E7325DD4  CN=Exchange2013A  960171662EB26116...








Keep reading